IC 

ELECTE 
DEC  2  7  1994 


I 


EVALUATION  OF  AN  AIR  FORCE 
INFORMATION  RESOURCES  MANAGEMENT  (IRM) 
OVERVIEW  COURSE 

THESIS 

Joseph  L.  Cox,  First  Lieutenant,  USAF 
Brenda  R.  Forcht,  Captain,  USAF 

AnT/GIR/LAR/94D-3 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


19941221  119 


AFIT/GIR/LAR/94D-3 


EVALUATION  OF  AN  AIR  FORCE 
INFORMATION  RESOURCES  MANAGEMENT  (IRM) 
OVERVIEW  COURSE 

THESIS 

Joseph  L.  Cox,  First  Lieutenant,  USAF 
Brenda  R.  Forcht,  Captain,  USAF 

AnT/GIR/LAR/94D-3 


Approved  for  public  release;  distribution  unlimited 


The  views  expressed  in  this  thesis  are  those  of  the  authors  and  do  not  reflect  the  official 
policy  or  position  of  the  Department  of  Defense  or  the  U.S.  Government. 


AFIT/GIR/LAR/94D-3 


EVALUATION  OF  AN  AIR  FORCE 
INFORMATION  RESOURCES  MANAGEMENT  (IRM) 

OVERVIEW  COURSE 

THESIS 

Presented  to  the  Faculty  of  the  Graduate  School  of  Logistics  and  Acquisition  Management 

of  the  Air  Force  Institute  of  Technology 
Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the 
Requirements  for  the  Degree  of 
Master  of  Science  in  Logistics  Management 


Joseph  L.  Cox,  B.B.A.  Brenda  R.  Forcht,  B.A. 

First  Lieutenant,  USAF  Captain,  USAF 


December  1994 


Approved  for  public  release;  distribution  unlimited 


Preface 


The  purpose  of  this  thesis  is  to  evaluate  an  Air  Force  information  resources 
management  (IRM)  overview  course.  This  study  included  a  quantitative  and  qualitative 
assessment  of  the  effectiveness  of  the  course.  Likewise,  the  research  provided  a  general, 
qualitative  evaluation  of  the  course  as  a  whole,  functioning  instructional  system. 

Many  people  contributed  mightily  to  the  successful  completion  of  this  research 
effort.  We  wish  to  thank  the  participants  in  the  pilot  administration  of  the  course.  Their 
inputs  set  the  stage  for  numerous  improvements  to  the  course.  With  regards  to  the  pilot 
administration  of  the  course,  we  would  also  like  to  thank  Major  Phil  McDowell  and 
Captain  Dave  Morgan  for  their  extensive  help  in  coordinating  the  whole  effort.  Likewise, 
we  would  like  to  thank  those  individuals  who  participated  in  our  reliability  testing.  Of 
course,  we  would  like  to  thank  our  research  advisors.  Dr.  Guy  Shane  and  Dr.  Kim 
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Abstract 

The  federal  government  has  passed  laws  and  issued  directives  recognizing  that 
information  is  a  valuable  national  resource.  To  help  assure  that  Air  Force  information 
managers  are  capable  of  implementing  policy  in  support  of  such  laws  and  directives,  the 
Secretary  of  the  Air  Force’s  Directorate  of  Information  Management  (SAF/AAI)  directed 
a  team  from  the  Air  Force  Institute  of  Technology  (AFIT)  to  design  and  constmct  an 
exportable  course  to  provide  an  overview  of  information  resources  management  (IRM) 
concepts  and  principles.  Such  an  IRM  overview  course  was  completed  in  the  Summer  of 
1994.  The  authors  evaluated  this  course  quantitatively  and  qualitatively.  During  a  pilot 
administration  of  the  course,  the  authors  used  a  quantitative  measurement  instmment  and 
a  pre-test/posttest  research  design  to  assess  effectiveness  of  the  instructional  system.  The 
authors  also  gathered  feedback  from  students  and  instructors  to  provide  a  summative 
evaluation  of  the  course  as  a  whole.  Based  on  test  scores  and  qualitative  feedback,  the 
authors  concluded  that  the  course  did  provide  a  useful  overview  of  IRM  concepts  and 
principles.  Additionally,  they  were  able  to  recommend  several  changes  to  the  course 
presentation,  content,  and  measurement  instrument. 
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EVALUATION  OF  AN  AIR  FORCE  INFORMATION  RESOURCES  MANAGEMENT 

(IRM)  OVERVIEW  COURSE 


I.  Introduction 


Background 

By  the  mid  1980s,  managers  realized  that  automation  alone  was  not  the  answer  to 
increased  productivity.  Management  began  to  view  information  as  a  resource  that  needs 
to  be  managed  for  increased  productivity,  and  automation  as  a  tool  to  provide  managers 
access  to  the  information  that  they  need  to  make  decisions.  Furthermore,  businesses 
recognized  that  solutions  to  organizational  productivity  problems  lay  not  in  how  much 
infoimation  can  be  automated,  but  rather,  in  what  information  was  necessary  to  support 
vital  business  processes.  A  business  process  is  "a  group  of  logically  related  tasks 
(decisions  and  activities)  that,  when  performed,  utilize  the  resources  of  the  business  to 
produce  definitive  results"  (Kane,  1986:26). 

As  a  result  of  the  newfound  importance  of  information,  various  methodologies 
were  developed  to  effectively  manage  organizational  information.  These  methodologies 
eventually  evolved  into  a  unique  management  sub-discipline  known  as  information 
resources  management  (IRM). 

The  Importance  of  IRM.  IRM  is,  in  its  most  general  sense,  the  management  of 
information  as  a  corporate  resource  (Newcomer  and  Caudle,  1986:29-31).  Organizations 
have  always  been  concerned  with  managing  corporate  resources  in  a  productive  manner. 
Historically,  the  most  important  corporate  resources  have  been  personnel,  capital,  and 


finances.  However,  over  the  past  two  decades,  information  has  come  to  hold  equal,  if  not 
superior,  status  to  these  crucial  resources  (Zwass,  1992:15). 

As  the  importance  of  information  and  data,  which  are  the  component  parts  of 
information,  became  more  obvious,  organizational  strategies  for  the  management  of 
information  developed.  Most  information  management  strategies  revolve  around  the 
ability  to  organize  mission-critical  data  into  corporate  databases  (Zwass  1992:16).  IRM 
strategies  provide  a  means  to  integrate  the  technical  specifications  for  data  storage  and 
retrieval  with  management  and  customer  needs  for  data  access.  Such  technical 
specifications  become  the  basis  for  information  systems,  whereas  management  and 
customer  needs  are  defined  by  organizational  transactions  and  processes.  Thus,  IRM  is  a 
mechanism  through  which  technical  capabilities  (information  technology)  and 
organizational  processes  are  matched  with  user  needs  to  benefit  the  organization 
(Newcomer  and  Caudle,  1986:29-31;  Zwass,  1992:16). 

Federal  Government  and  IRM.  Although  typically  associated  with  private  sector 
initiatives,  IRM  principles  also  apply  to  government  organizations  (Corbin,  1993:26).  In 
fact,  the  federal  government  should  be  especially  careful  to  effectively  manage  its 
information  resources.  Because  it  is  the  basis  for  virtually  every  transaction  conducted  by 
the  government,  "information  is  to  government  what  manufactured  goods  are  to  private 
industry"  (Johnson,  1992:6).  Indeed,  "the  federal  government  is  the  nation’s  largest  single 
producer,  consumer,  and  disseminator  of  information"  (GSA,  1993:1-1).  Therefore,  it  is 
no  surprise  that  in  light  of  recent  government  initiatives  to  cut  spending  and  streamline 
business  processes,  adoption  of  IRM  concepts  has  become  one  of  the  prime  mandates  of 
the  federal  legislature  and  the  executive  branch.  The  importance  of  IRM  has  led  the 
government  to  establish  commissions,  launch  programs,  pass  laws,  and  perform  studies  to 
examine  federal  IRM  activities  (GSA,  1993:1-1). 
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Legislative  Iniriatives.  Although  an  abundance  of  federal  IRM  legislation 
exists,  the  Paperwork  Reduction  Act  of  1980,  the  Office  of  Management  and  Budget 
(0MB)  Circular  A- 130,  and  the  Federal  Information  Resource  Management  Regulations 
(FIRMR)  are  some  of  the  most  important  legislative  documents  addressing  federal 
implementation  of  IRM  principles  (Johnson,  1992:8-9).  Together,  these  legislative 
directives  outline  organizational  IRM  responsibilities,  policies,  and  procedures  for  federal 
government  agencies.  As  such,  these  legislative  documents  require  federal  agencies  to 
manage  information  in  an  efficient,  effective,  and  economical  manner,  essentially  making 
IRM  an  agency-wide  function.  Today,  federal  managers  "must  address  all  of  the  planning, 
budgeting,  organizing,  directing,  training,  and  controlling  functions  associated  with 
information"  (Johnson,  1992:6). 

Executive  Initiatives.  In  addition  to  the  legislative  backing  for  IRM, 
President  Clinton  and  Vice  President  Gore  have  recently  introduced  two  initiatives  that 
spotlight  the  executive  branch's  commitment  to  IRM  principles.  First,  the  executive 
branch  initiated  a  National  Performance  Review  (NPR)  to  identify  inefficient  business 
processes  in  need  of  process  reengineering  (Corbin,  1993:26).  Process  reengineering  is  a 
technique  pioneered  by  business  consultant  Michael  Hammer  which  uses  the  “power  of 
modem  information  technology  to  radically  redesign.. .business  processes  in  order  to 
achieve  dramatic  improvements”  in  organizational  effectiveness  (Hammer,  1990:104). 
Through  this  reengineering  effort,  the  President  hopes  to  "challenge  the  basic  assumptions 
of  every  program,  asking  does  it  work;  does  it  provide  quality  service;  does  it  encourage 
innovation  and  reward  hard  work"  (Grassley,  1993:168).  Further,  a  main  goal  of  the  NPR 
is  to  allow  the  executive  branch  to  begin  "getting  rid  of  12  percent  of  the  government's 
civilian  employees-252,000  jobs-by  cutting  layers  of  management”  (Shoop,  1994:10). 
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Additionally,  through  reengineering  efforts,  the  Department  of  Defense  hopes  to  reduce  its 
administrative  spending  by  as  much  as  $70  billion  (Green,  1994:84). 

Second,  the  White  House  introduced  the  National  Information  Infrastructure  (Nil) 
Agenda  for  Action  which  outlines  plans  for  the  development  of  a  nationwide  network  of 
networks  to  insure  universal  access  to  federal  information  resources  (The  White  House, 
1993a).  The  Nil  Agenda  for  Action  "dramatically  sketched  a  vision  of  the  near  future,  in 
which  a  web  of  advanced  communications  networks  and  computers  would  bring  vast 
amounts  of  information  and  greatly  improved  services  to...virtually  every  citizen"  (IITF, 
1994). 


Focus  on  People.  Clearly,  IRM  is  an  important  issue  for  the  public  and 
private  sectors.  However,  until  recently,  the  federal  sector  had  been  much  slower  than 
industry  to  adopt  IRM  principles.  Consequently,  the  government  recently  committed  to 
making  IRM  strategic  planning  one  of  its  top  priorities.  Moreover,  because  IRM 
initiatives  must  be  supported  and  carried  out  by  highly  skilled  personnel,  the  federal 
government  has  made  the  acquisition,  education,  and  training  of  IRM  professionals  one  of 
the  tenets  of  its  overall  strategic  planning  model  (GSA,  1993:2-3). 

Department  of  the  Air  Force  and  IRM.  In  compliance  with  federal  IRM  directives, 
the  Department  of  the  Air  Force  has  recently  focused  on  the  role  of  IRM  in  the 
performance  of  its  mission.  The  Air  Force  considers  effective  management  of  information 
resources  the  "key  to  battlefield  interoperability  and  aerospace  domination"  (SAF, 

1993:1).  In  fact,  the  Air  Force  has  recently  committed  to  incorporating  IRM  doctrine  into 
the  overall  doctrine  of  the  Air  Force,  AFMAN  1-1,  Basic  Aerospace  Doctrine  of  the 
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United  States  Air  Force.  The  new  version  of  the  Air  Force  doctrine  document  will,  for  the 
first  time,  incorporate  information  life-cycle  management  concepts,  a  main  tenet  of  IRM. 
Further,  current  research  at  the  Air  Force  Institute  of  Technology  (AFTT)  is  expected  to 
produce  a  list  of  IRM  tenets  to  be  the  basis  for  inclusions  to  future  versions  of  Air  Force 
doctrine  (Bowser,  1994). 

Acknowledging  the  key  role  that  IRM  professionals  must  play  in  support  of  the 
mission,  the  Air  Force  has  developed  five  strategic  goals  related  to  the  management  of 
information,  as  well  as  accompanying  objectives  and  task  plans  outlining  how  these  goals 
are  to  be  accomplished.  The  five  goals  follow: 

Goal  1:  Enhance  readiness  to  support  the  USAF  war  and  contingency  mission 

Goal  2:  Improve  information  management  support  to  commanders 

Goal  3:  Develop  information  managers  for  changing  environments 

Goal  4:  Improve  the  methods  for  managing  information  throughout  its  lifecycle 

Goal  5:  Improve  information  management  (IM)  resource  management. 

(SAF,  1993:8-9) 

Although  each  of  these  goals  is  vitally  important  to  the  overall  mission  of  the  Air 
Force  information  management  community,  the  development  of  information  managers  for 
changing  environments  is  a  prime  focus  given  recent  federal  government  initiatives. 
Presently,  Air  Force  information  managers  perform  duties  which  have  not  historically 
embraced  the  principles  of  IRM.  In  fact.  Air  Force  information  managers  often  do  not 
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have  the  skills  necessary  to  properly  manage  information  resources  (SAF,  1993:51). 
Information  managers  without  IRM  skills  cannot  carry  out  their  primary  mission- "to 
assist  functional  customers  in  identifying  their  information  needs"  (SAF,  1993:55). 
Consequently,  the  Air  Force  Directorate  of  Information  Management  (SAF/AAI)  has 
identified  a  specific  need  to  expand  educational  and  training  opportunities  for  information 
managers.  Air  Force  leadership  intends  that  information  managers  will  hasten  the 
integration  of  IRM  principles  at  every  organizational  level  (SAF,  1993:55). 

In  one  effort  to  satisfy  this  training  requirement,  SAF/AAI  tasked  a  team  from 
AFIT  to  design  and  construct  an  exportable  IRM  course  tailored  to  familiarize  mid-level 
Air  Force  information  managers  with  IRM  concepts  and  principles.  It  should  be  noted 
that  the  researchers  for  this  study  were  a  part  of  the  team  that  developed  the  course.  The 
first  version  of  this  course  was  recently  completed.  The  seven-module  course  was 
designed  to  provide  an  overview  of  IRM  concepts  and  principles.  Table  1  describes  the 
IRM  subject  matter  contained  in  each  module  of  the  course. 


Table  1 

IRM  Course  Content 


Module 

Content 

1 

Introductory  IRM  concepts  and  terms 

2 

Key  federal  IRM  laws  and  organizations 

3 

Corporate  Information  Management  goals  and  policies 

4 

Strategic  planning  process  and  IRM  applications 

5 

Business  process  improvement  methodology 

6 

Systems  development  life-cycle  concepts 

7 

Information  engineering  methodology 

6 


General  Issue 


Before  fielding  the  IRM  course  developed  by  the  AFIT  project  team,  SAF/AAI 
directed  the  course  be  evaluated  in  accordance  with  AFMAN  1-1.  AFMAN  1-1  states 
that  “people  are  the  decisive  factor  in  war,”  and  training  and  education  are  necessary  to 
effectively  prepare  people  to  perform  their  mission.  Likewise,  this  doctrine  dictates  that 
all  training  and  education  should  be  vigorously  evaluated  (DAF,  1992:18).  AFMAN  36- 
2234,  Instructional  System  Development  (ISD)  provides  a  mechanism  for  satisfying  this 
evaluation  requirement. 

ISD  Functions.  The  ISD  process  exists  within  the  larger  framework  of  the  five 
main  functions  of  a  total  instructional  system.  From  the  most  general  level,  a  total 
instmctional  system  is  composed  of  the  following  interrelated  functions:  management, 
support,  administration,  delivery,  and  evaluation.  Figure  1  shows  how  these  functions 
mesh  together  and  emphasizes  the  central  role  which  evaluation  plays  in  the  development 
of  instraction.  The  scope  of  each  of  the  functions  is  described  below. 

•  Management  -  the  function  of  directing  or  controlling  instructional  systems 
development  and  operations 

•  Support  -  the  function  of  maintaining  all  parts  of  the  system 

•  Administration  -  the  function  of  day-to-day  processing  and  record  keeping 

•  Delivery  -  the  function  of  bringing  instruction  to  students 

•  Evaluation  -  the  function  of  gathering  feedback  data  through  formative, 
summative,  and  operational  evaluations  to  assess  system  and  student 
performance.  (DAF,  1993b:  12-13) 
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The  ISD  process  is  composed  of  four  phases;  analysis,  design,  development,  and 
implementation  (DAF,  1993b:  14).  These  phases  are  described  in  detail  and  the  complete 
ISD  model  is  presented  in  the  discussion  of  methodology  in  Chapter  II  of  this  paper. 

From  the  description  of  ISD  provided  to  this  point,  it  is  important  to  reahze  that 
“evaluation  is  a  central  function  that  takes  place  at  every  phase”  of  ISD  (DAF,  1993b;  14). 
Consequently,  rigorous  evaluation  is  the  cornerstone  of  effective  instructional  system. 


Figure  1.  ISD  Systems  Functions  (DAF,  1993b:  12) 

ISD  and  Evaluation.  Evaluation  is  the  act  of  providing  "feedback  leading  to  a 
successful  outcome  defined  in  practical ,  concrete  terms"  (Isaac  and  Michael,  1985:2). 

The  purpose  of  instructional  evaluation  is  to  improve  instructional  systems  by  measuring 
performance  against  established  standards  (Bloom,  1971:7).  Without  evaluation,  one  has 
no  method  of  assessing  the  value  of  an  instructional  system.  Three  types  of  evaluation  are 
germane  to  ISD:  formative  evaluation,  summative  evaluation,  and  operational  evaluation 
(DAF,  1993a:7). 

Formative  Evaluation.  Formative  evaluation  is  the  process  of  keeping  an 
instructional  system  tme  to  its  design.  Such  evaluation  is  focused  on  providing  feedback 
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for  making  mid-course  corrections  to  the  system  (Isaac  and  Michael,  1985:5;  Foley, 
1971:804-805).  Formative  evaluation  includes  the  process  and  product  evaluations 
conducted  during  the  analysis  and  design  phases  of  ISD  and  validation  through  small 
group  tryouts  during  the  development  phase  of  ISD  (DAF,  1993a:7). 

Summative  Evaluation.  Summative  evaluation  provides  a  "general 
assessment  of  the  degree  to  which...larger  outcomes  have  been  obtained  over  the  entire 
course  of  instruction"  (Bloom,  1971:61).  This  type  of  evaluation  might  include 
operational  tryouts  of  an  instructional  system.  Summative  evaluation  is  typically 
conducted  as  the  last  step  of  the  validation  process  during  the  development  phase  of  ISD 
(DAF,  1993a:7). 

Operational  Evaluation.  Operational  evaluation  is  basically  on-going 
evaluation  of  an  instructional  system,  after  it  has  been  adopted  in  the  field.  The  purpose  of 
such  evaluation  is  to  enhance  an  instructional  system  during  full-scale  operations  (DAF, 
1993b:  187).  As  such,  operational  evaluation  includes  periodic  internal  and  external 
evaluation  of  an  operational  instructional  system  (DAF,  1993a:7). 

Specific  Purpose 

SAF/AAI  identified  a  need  for  delivery  of  IRM  education  and  training  to  all  Air 
Force  information  managers.  In  an  effort  to  satisfy  one  part  of  this  reqiurement,  a  team 
from  AFTT  developed  an  exportable  IRM  course  for  mid-level  Air  Force  information 
managers.  Our  purpose  in  this  research  was  to  evaluate  this  course. 
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Research  Objective 


The  primary  objective  of  our  research  was  to  evaluate  the  course  as  an 
instructional  system.  In  support  of  this  objective,  we  needed  to  met  two  sub-objectives. 
First,  we  had  to  determine  the  basis  for  evaluating  the  course.  Second,  we  needed  to 
develop  measurement  tools  to  gather  the  data  to  be  used  in  making  an  evaluation. 

Thesis  Overview 

Chapter  n  describes  the  methodology  used  in  this  research.  It  presents  and 
explains  our  research  design  and  measurement  instruments.  Chapter  III  describes  and 
analyzes  data  obtained  from  our  evaluation  of  the  IRM  course.  As  such.  Chapter  III 
provides  quantitative  and  qualitative  data  regarding  the  effectiveness  of  the  course. 
Finally,  Chapter  IV  contains  conclusions,  recommendations,  and  suggestions  for  further 
research. 
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n.  Methodology 


Introduction  of  Methods 

In  order  to  meet  our  research  objective  and  sub-objectives,  we  employed  a 
combination  of  methodologies.  We  conducted  our  research  within  the  framework  of  the 
ISD  model.  After  outlining  the  methodology  associated  with  the  ISD  model  and  our 
subject  course’s  development  in  relation  to  the  model,  we  will  present  our  research  design, 
discussing  the  development  of  our  measurement  instrument  and  the  administration  of  our 
experimental  treatment. 

ISD  Model 

The  ISD  model  is  designed  to  aid  in  the  development  of  effective,  cost-efficient 
instruction.  The  overall  model  has  three  major  parts:  systems  functions  (described  in 
Chapter  I),  ISD  phases,  and  quality  improvement  (DAF,  1993a:3).  Figure  2  provides  a 
view  of  the  entire  ISD  model.  It  depicts  the  systems  functions  and  ISD  phases  embedded 
within  the  quality  improvement  process.  The  following  sections  briefly  discuss  the  ISD 
phases,  the  quality  improvement  aspects  of  the  ISD  model,  and  how  the  course  fits  into 
the  model. 

ISD  Phases.  The  ISD  process  is  composed  of  four  distinct  phases:  analysis, 
design,  development,  and  implementation.  The  following  sections  describe  each  of  these 
phases. 
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Analysis  Phase.  In  this  phase,  the  instructional  designer  analyzes  the 
requirements  which  the  instruction  is  seeking  to  fulfill.  The  designer  seeks  to  link 
performance  requirements  to  knowledge,  skills,  and  abilities  that  can  be  presented  through 
instmction  (DAF,  1993a:6). 


Design  Phase.  In  this  phase,  “the  instructional  designer  develops  a  detailed 
plan  of  instmction”  (DAF,  1993a:6).  This  plan  includes  instructional  methods,  media,  and 
strategies.  Existing  instmctional  materials  are  reviewed  to  determine  their  applicability  for 
the  instmction  under  development.  Ideally,  this  phase  also  includes  the  development  of 
objectives  and  tests  (DAF,  1993a:6). 
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Development  Phase.  In  this  phase,  the  instructional  designer  develops 
student  and  instructor  material.  Additionally,  the  designer  validates  each  module  of 
instruction  and  its  supporting  material.  This  validation  should  include:  internal  review 
(formative  evaluation);  small  group  tryouts  (formative  evaluation);  and  field  tryouts  of  the 
system  (summative  evaluation).  Revisions  are  then  made  to  the  course  based  on  this 
validation  process,  and  the  instructional  materials  are  finalized  (DAF,  1993a:6). 

Implementation  Phase.  In  this  phase,  the  instmctional  system  is  ready  to  be 
implemented.  As  such,  it  is  fielded  under  normal  operating  conditions.  To  assure  the 
continuing  quality  of  the  instruction  and  material,  internal  and  external  evaTuations  are 
conducted  to  provide  feedback  for  improvements  to  the  system  (DAF,  1993a:6). 

ISD  Model  and  Quality  Improvement.  “Quality  improvement  is  the  continuous, 
organized  creation  of  beneficial  change  to  the  instructional  system”  (DAF,  1993a:8). 
Quality  improvement  is  intended  to  foster  continuous  improvement  in  the  instructional 
process  and  the  product  of  that  process.  In  general,  the  ISD  process  is  reliant  on  many 
quality  improvement  tenets,  including:  “the  customer  defines  quality”,  “know  your 
mission”,  “know  your  customer”,  “set  standards”,  and  “evaluate  quality  constantly” 

(DAF,  1993a:  8-9).  As  such,  the  principles  of  quality  improvement  are  implemented  in  the 
ISD  process. 

ISD  and  the  IRM  Course.  Although  the  course  we  evaluated  had  completed  the 
analysis  phase, of  ISD,  it  had  not  completed  the  design  or  development  phases. 

Specifically,  the  student  and  instructor  lesson  materials  had  been  developed,  the 
presentation  media  selected,  and  slides  produced.  An  internal  review  of  the  course 
materials  for  accuracy  (formative  evaluation)  had  been  completed  by  the  course 
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developers  and  local  content  experts.  In  addition,  the  course  designers  had  completed  a 
small-group  tryout  (formative  evaluation)  by  presenting  several  modules  of  instmction 
from  the  course  to  a  group  of  graduate  IRM  students  from  AFIT.  However,  the  designers 
had  not  developed  a  test  for  the  instructional  system,  nor  had  they  completed  a  field  tryout 
of  the  whole  system  (summative  evaluation).  The  evaluative  focus  of  our  research 
therefore  was  to  develop  a  test  for  the  instructional  system  and  to  conduct  summative 
evaluation. 

Research  Design 

The  main  thrust  of  our  research  was  to  evaluate  the  IRM  course.  We  chose  a  one- 
group  pretest-posttest  research  design  as  the  primary  mechanism  for  determining  the 
effectiveness  of  the  course  and  to  provide  a  fomm  for  other  summative  evaluation.  This 
pre-experimental  design  allowed  for  a  simple,  direct  comparison  of  student  scores  before 
and  after  the  administration  of  the  experimental  treatment,  the  IRM  course  instruction 
(Campbell  and  Stanley,  1963:7). 

We  analyzed  the  difference  in  mean  scores  within  each  module  between 
administrations  via  a  paired  difference  test.  To  perform  this  test,  we  used  the  paired  t-test 
function  within  Statistix,  a  statistical  software  program  (Statistix,  1992).  In  performing 
this  test,  we  looked  for  evidence  that  the  mean  scores  within  each  module  from  the  first 
administration  of  the  measurement  instrument  would  be  significantly  lower  than  the  mean 
scores  within  each  module  from  the  second  administration  of  the  measurement  instrument 
(McClave  and  Benson,  1991:424). 

In  order  to  perform  this  test,  we  made  two  assumptions  about  the  population  of 
differences.  First,  we  assumed  that  the  relative  frequency  distribution  of  the  population  of 
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differences  is  normal.  Second,  we  assumed  that  these  differences  are  randomly  selected 
from  the  population  of  differences  (McClave  and  Benson,  1991:424). 

Measurement  Instruments 

We  developed  three  instraments  with  which  to  evaluate  the  effectiveness  of  the 
course.  First,  we  devised  two  qualitative  instruments  to  capture  instructor  and  student 
perceptions  of  the  course.  Second,  we  developed  a  quantitative  instrument  to  measure  the 
effect  of  the  treatment  (IRM  course)  on  the  students.  The  following  sections  present 
these  instruments  and  discuss  their  development. 

Qualitative  Instruments.  We  provided  instructors  and  students  with  the 
opportunity  to  assess  the  various  factors  impacting  the  effectiveness  of  the  IRM  course. 
The  student  feedback  form  is  found  in  Appendix  A,  and  the  instructor  feedback  form  is 
found  in  Appendix  B.  Both  groups,  instructors  and  students,  were  asked  to  comment  on 
the  relevance  of  course  content,  the  usefulness  of  course  materials,  and  the  extent  to 
which  the  course  met  its  overall  objective  (which  was  to  familiarize  students  with  IRM 
concepts  and  principles).  Additionally,  the  students  were  asked  to  comment  on  the  quality 
of  instructional  delivery. 

Quantitative  Instrument.  In  order  to  gather  the  data  need  to  summatively  evaluate 
the  IRM  course,  we  needed  an  instrument  to  quantitatively  measure  student  attainment  of 
the  behavior  specified  by  the  course  objective  (DAF,rl993b:77).  In  this  case,  we  needed 
to  measure  the  level  of  familiarity  with  the  material  presented  in  the  IRM  course.  Using 
such  an  instrument  within  the  structure  of  our  research  design,  we  hoped  to  gain  a 
measure  of  the  effectiveness  of  the  course.  To  this  end,  we  developed  an  objective 
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measurement  instrament  to  evaluate  the  IRM  course.  Specifically,  we  determined  that  a 
multiple-choice  format  test  instrument  was  the  most  appropriate  for  our  purpose. 

Characteristics  of  Multiple-Choice  Tests.  A  multiple-choice  test  was 
inherently  well  suited  for  our  purpose.  Aiken  described  the  following  characteristics  of 
multiple-choice  tests  which  were  particularly  important  for  our  study: 


•  Versatility,  multiple-choice  tests  can  measure  both  simple  and  complex 
objectives  at  almost  all  grade  levels  and  in  all  subject  areas 

•  More  adequate  sampling,  multiple-choice  tests  can  sample  the  domain  of 
abilities  more  satisfactorily  than  essay  items  and  almost  all  other  objective 
items 

•  Less  susceptiblity  than  true-false  items  to  both  guessing  and  response  sets 

•  Objectivity  of  scoring,  multiple-choice  tests  can  be  scored  accurately  and 
rapidly  by  a  clerk  (the  grader  need  not  have  special  knowledge  of  the  subject 
matter) 

•  The  provision  of  diagnostic  information  by  an  analysis  of  responses  to  item 
distracters 

•  Greater  reliability  than  other  test  items,  multiple-choice  tests  are  both  objective 
and  less  susceptible  to  guessing  effects.  (Aiken,  1987:44) 


Pool  of  Questions.  We  then  developed  a  pool  of  test  questions  that  were 
likely  to  accurately  measure  the  participants’  achievement  of  course  objectives.  Some 
questions  were  pulled  from  established  test  banks.  However,  when  established  test  banks 
with  appropriate  material  were  not  avmlable,  we  used  the  course  objectives  to  develop 
questions  directly  from  the  course  material.  We  originally  prepared  185  multiple-choice 
items  which  covered  the  entire  range  of  course  content. 
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Wood  suggests  basing  test  items  on  fundamental  concepts,  purposes,  causal 
relationships,  and  extensions  of  applications  to  new  problems,  rather  than  on  facts 
acquired  by  rote  memory  or  inconsequential  details  (Wood,  1960:44-50).  We  made  a 
great  effort  to  accomplish  these  goals  in  developing  the  test  items.  Additionally,  we 
followed  many  of  Wood’s  suggestions  in  constructing  test  items.  Most  items  were  written 
as  direct  questions,  although  a  few  were  incomplete  statements  which  required  the  test 
taker  to  complete  a  thought.  We  tried  to  be  economical  in  our  use  of  language,  so  as  to 
reduce  reading  time  and  complexity  for  test  takers.  Most  items  were  presented  in  a 
positive  form  with  emphasis  on  any  negative  aspects  of  the  items  to  avoid  confusion  on 
the  part  of  the  test  takers  (Wood,  1960:44-50). 

We  followed  Wood’s  advice  when  devising  the  alternatives  for  each  item  as  well. 
We  tried  to  ensure  the  alternatives  were  plausible  and  followed  grammatically  from  the 
statement  of  the  problem.  We  made  alternatives  parallel  in  grammatical  form  and  length 
to  avoid  giving  clues  to  the  correct  response.  None  of  the  above,  All  of  the  above,  and 
alternatives  indicating  more  than  one  correct  alternative  were  used  in  some  test  items. 
However,  whenever  possible,  we  provided  one  correct  response  choice  and  a  full  set  of 
plausible,  but  incorrect,  alternatives  to  serve  as  distracters.  Alternatives  were  randomly 
placed  using  a  random  number  table  except  in  situations  where  the  alternatives  logically 
belonged  in  a  set  order  (Wood,  1960:50-60). 

Validity.  Validity  customarily  refers  to  the  extent  to  which  an  instrument 
measures  that  which  its  user  intends  to  measure  (Lindquist,  1942:213).  According  to  this 
concept,  validity  is  determined  by  comparing  what  the  instrument  measures  against  what  it 
ought  to  measure.  For  the  purpose  of  our  research  content  validity  and  format  bias  (which 
could  affect  validity),  were  of  special  concern.  Content  validity  refers  to  the  extent  to 
which  a  test  includes  a  representative  sample  of  the  universe  of  content  and  objectives  for 
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the  course  or  subject  field  which  is  being  measured  (Green,  1970:64).  Format  bias,  as 
used  here,  refers  to  the  extent  to  which  the  test  items  were  formatted  in  a  manner  that 
might  be  ambiguous  to  test  takers. 

To  assure  we  achieved  content  and  avoided  format  bias  within  our  test,  the 
individual  test  items  from  our  pool  of  questions  were  evaluated  by  local  subject  matter 
experts.  Specifically,  each  test  item  was  reviewed  by  at  least  two  content  and  format 
experts. 


Qualification  of  Subject  Matter  Experts.  A  subject  matter  expert  is 
"a  person  who  has  high-level  knowledge  and  skill"  in  a  particular  subject  area  (DAF, 
1993b:  188).  The  item  reviewers  met  this  definitional  standard  as  subject  matter  experts. 
Specifically,  the  two  psychometrics  (format)  evaluators  hold  doctoral  degrees  and  have 
extensive  practical  experience  in  the  field  of  psychometrics.  Of  the  five  IRM  evaluators, 
four  hold  doctoral  degrees  in  IRM  or  a  related  field.  The  other  IRM  subject  matter  expert 
holds  a  masters  degree  in  IRM  and  is  a  practicing  information  resources  manager. 

Content  Validity.  With  respect  to  content  validity,  because  the 
course  is  composed  of  seven  modules  which  focus  on  different  disciplines  within  the  field 
of  IRM,  different  combinations  of  evaluators  were  used  for  each  module.  Each  content 
expert  was  asked  to  self-determine  which  sections  he  felt  most  confident  in  evaluating. 
Based  on  the  responses  from  the  evaluators,  we  assigned  each  content  expert  only  those 
items  which  were  designed  to  test  familiarity  with  material  from  the  module(s)  in  which  he 
felt  most  knowledgeable.  The  test  items  were  evaluated  as  to  whether  they  actually 
measured  the  concepts  outlined  within  the  objectives  for  a  particular  module  and 
adequately  represented  the  content  of  the  subject  area  from  which  the  module’s  material 
was  drawn.  In  cases  where  the  experts  had  a  difference  of  opinion,  each  evaluator  was 
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informed  of  the  opinions  of  the  remaining  experts  and  a  consensus  was  reached. 
Concerning  content  validity,  we  selected  only  those  test  items  found  to  be  valid  by  the 
content  experts. 

Format  Bias.  Format  bias  was  assessed  by  having  two 
psychometrics  experts  review  the  structure  of  each  test  item  that  survived  content 
validation.  As  with  our  content  validation  process,  when  these  two  experts  disagreed  as 
to  the  existence  of  bias  in  the  format  of  an  item,  each  was  informed  of  the  other’s  opinion, 
and  a  consensus  was  reached.  Only  those  items  which  passed  this  formating  process  were 
included  in  our  initial  160-question  measurement  instmment. 

Selecting  Items  for  the  Measurement  Instrument.  To  ascertain  which  items 
would  be  included  in  the  final  measurement  instrument,  a  test  containing  all  160  test  items 
remaining  in  the  pool  was  administered  to  28  AFIT  graduate  students  with  knowledge  and 
skills  that  closely  resemble  the  knowledge  and  skills  that  subjects  taking  the  IRM  course 
should  have  upon  completion  of  the  course.  This  test  is  found  in  Appendix  C.  Based  on 
the  results  of  this  testing,  we  used  a  criterion  based  on  p-values  to  select  those  items 
which  would  be  most  appropriate  for  inclusion  in  the  final  measurement  instrument. 
F-values  are  the  proportion  of  participants  responding  correctly  to  an  item.  F-values 
measure  item  difficulty  (Dick  and  Hagerty,  1971:32).  Maximum  reliability  occurs  when 
item  difficulty  is  50  percent  (Helmstadter,  1964:169;  Guion,  1965:49).  In  an  effort  to 
maximize  the  reliability  of  the  final  measurement  instrument,  we  selected  items  with  p- 
values  ranging  from  .25  to  .75. 

Due  to  the  limited  time  available  for  course  presentation,  of  which  testing  was  a 
part,  we  decided  to  allot  approximately  30  minutes  of  course  time  for  each  test 
administration.  Reliability  testing  had  shown  that  students  were  taking  approximately  30 
seconds  to  complete  each  question.  Based  on  this  finding  and  the  time  available  for 
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testing,  we  determined  the  final  test  would  contain  60  items.  This  test  is  found  in 
Appendix  D.  Although  longer  tests  are  generally  more  reliable  than  shorter  tests,  we  were 
forced  to  sacrifice  the  opportunity  for  better  coverage  of  the  course  material  by  the  above- 
mentioned  time  restrictions. 

Adequacy  of  Content  Sampling.  Adequate  coverage  of  the  course 
material  is  a  vital  consideration  in  the  development  of  a  measurement  instrument  (DAF, 
1993b:83).  In  order  to  ensure  that  the  test  covered  the  course  material  appropriately,  we 
determined  the  relative  importance  of  each  of  the  course’s  seven  modules.  Specifically, 
the  course  developers  made  a  subjective  judgment  as  to  the  relative  importance  of  each 
module.  This  importance  was  expressed  as  the  percentage  each  module  contributed  to  the 
course  as  a  whole.  Differences  of  opinion  were  openly  discussed,  and  a  consensus  was 
reached  on  the  final  percentages.  These  percentages,  listed  in  Table  2,  were  used  to 
determine  the  number  of  questions  from  each  course  module  that  would  be  included  in  the 
measurement  instrument. 

Our  selection  criteria,  based  on  choosing  items  with  a  p- value  between  .25  and  .75 
and  the  content  sampling  percentages  listed  in  Table  2  did  not  yield  a  complete  set  of  60 
items  for  the  measurement  instrument.  Consequently,  we  selected  additional  items  outside 
of  the  p-value  criterion  range  to  bring  the  total  number  of  items  to  60. 


20 


Table  2 


Relative  Importance  of  Modules 


Module 

Percent  of  Test  Items 

1 

20 

2 

7 

3 

3 

4 

10 

5 

27 

6 

20 

7 

13 

Reliability  of  Instrument.  Reliability  can  be  defined  as  the  degree  of 
consistency  between  two  measiues  of  the  same  thing  (Mehrens  and  Lehmann,  1991:249). 
Test  reliability  may  be  defined  as  the  degree  to  which  errors  of  measurement  influence  test 
scores.  Errors  of  measurement  are  those  influences  on  test  scores  that  are  not  related  to 
the  attribute  being  measured  by  a  test  scores.  It  is  important  to  keep  clearly  in  mind  that 
test  reliability  deals  with  accuracy  of  measurement;  it  is  only  indirectiy  a  measure  of  the 
usefulness  or  meaningfulness  of  that  measure  (Womer,  1968:30). 

We  used  an  internal-consistency  estimate  of  reliability  to  determine  the  reliability 
of  the  instrument.  Specifically,  the  Kuder-Richardson  formula  20  was  used  to  estimate  the 
instrument’s  reliability.  This  formula,  generally  referred  to  as  K-R  20,  follows: 
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(1) 


r,t  =  {k/k-\)  (Oo^  -  Spfli/Oo^) 


where 

k  =  number  of  items  in  the  test 

Pi  =  proportion  of  participants  responding  correctly  to  item  i 
Qi  =  1  -  p  (or  proportion  of  participants  responding  incorrectly) 

Go  =  test  variance 

'LpiQi  -  sum  of  p  times  q  for  all  items  (Dick  and  Hagerty,  1971:32). 


K-R  20  (equation  1)  measures  the  homogeneity  of  the  items  in  a  test  and  does  not  assess 
stability  over  time.  Thus,  it  may  be  interpreted  in  the  same  fashion  one  would  interpret 
Cronbach’s  alpha  reliability  coefficient  (Womer,  1968:37).  The  estimates  of  reliability  for 
the  questions  in  our  measurement  instrument,  broken  out  by  module,  are  displayed  in 
Table  3. 


Standard  Error  of  Measurement.  The  standard  error  of  measurement  is  an 
additional  concept  which  is  important  in  any  discussion  of  reliability.  The  standard  error 
of  measurement  is  a  statistic  that  yields  a  score  in  the  same  units  as  the  raw  score  of  a  test. 
The  interpretation  of  the  standard  error  of  measurement  is  similar  to  that  of  the  standard 
deviation  of  a  set  of  test  scores.  The  formula  used  to  compute  the  standard  error  of 
measurement  follows: 


^meas 


(2) 


where 

Go=  standard  deviation  of  the  test  scores 

rtt  =  reliability  estimate  of  the  test  (Dick  and  Hagerty,  1971:39). 
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Actual  Values.  The  complete  60-item  measurement  instrument  had  an  internal 


consistency  estimate,  or  reliability,  of  0.58  with  a  standard  error  of  measurement  of  3.26. 
This  reliability  estimate  was  derived  from  the  values  obtained  from  the  34  participants' 
scores  on  the  60  items  included  in  the  final  measurement  instrument.  The  following  table 
summarizes  the  internal  consistency  estimates  and  standard  errors  of  measurement  for 
each  module. 


Table  3 


Instrument  Reliability  Estimates  and  Standard  Errors  of  Measurement 


Module 

1 

Module 

2 

Module 

3 

Module 

4 

Module 

5 

Module 

6 

Module 

7 

Reliability 

Estimate 

.36 

.09 

-.41* 

.38 

.72 

.07 

-.80* 

Std  Error  of 

Measurement 

1.551 

.9561 

.7405 

1.0122 

1.7938 

1.6476 

1.3116 

Maximum 

Possible  Score 

12 

4 

2 

6 

16 

12 

8 

‘Negative  estimates  are  clearly  artifacts  of  a  lack  of  variance  among  the  small  sample;  in  all 
probability  due  to  small  sample  size. 


Administration  of  Course 


The  IRM  course  was  administered  over  a  three-day  period,  24  October  through  26 
October  1994,  at  HQ  AFMC,  Wright-Patterson  Air  Force  Base,  Ohio.  We  administered 
the  final  measurement  instrument  to  the  six  participants  during  the  first  hour  of  the  first 
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day  of  instruction.  Then,  after  the  students  had  been  exposed  to  the  entire  course  of 
instruction,  we  re-administered  the  measurement  instrument  during  the  last  hour  of  the 
final  day  of  instruction.  The  students  attended  class  from  0800  to  1200  hours  each  of  the 
three  days,  receiving  instruction  from  a  different  instractor  each  day. 

Subjects.  The  six  student  participants  were  drawn  from  a  sample  which  reasonably 
represented  the  target  population  for  the  instructional  system.  The  course  was  designed  to 
target  mid-level  Air  Force  information  resources  managers.  Each  of  the  subjects 
performed  duties  which  could  reasonably  be  included  within  the  field  of  IRM.  Likewise, 
each  student  held  positions  which  could  be  considered  to  fall  into  the  category  of  mid¬ 
level  management. 

Instructors.  The  three  instructors  were  drawn  directly  from  the  available  pool  that 
is  expected  to  form  the  population  of  instructors  for  the  course.  Specifically,  each 
instructor  was  a  student  or  graduate  of  the  AFIT  Graduate  IRM  program. 

Summary  of  Methods 

The  design  for  this  research  was  fitted  within  the  framework  of  the  Air  Force  ISD 
model.  Working  within  this  structure,  we  developed  a  measurement  instrument  and 
evaluated  the  IRM  course  quantitatively  using  a  one-group  pretest-posttest  pre- 
experimental  design.  Additionally,  we  qualitatively  evaluated  the  IRM  course  through  the 
use  of  open-ended  survey  devices  administered  to  the  students  and  instructors. 


24 


ni.  Results  and  Discussion 


Introduction 

This  section  presents  an  analysis  of  the  data  gathered  within  the  scope  of  our 
research  and  a  discussion  of  findings  resulting  from  this  analysis.  The  first  section 
provides  analysis  and  discussion  of  qualitative  data  supplied  by  the  students  and 
instructors.  The  second  section  provides  analysis  and  discussion  of  quantitative  data 
gathered  through  the  use  of  our  test  instrument. 

Analysis  and  Discussion  of  Qualitative  Data 

We  asked  the  students  and  instructors  who  participated  in  the  administration  of  the 
IRM  course  to  evaluate  various  aspects  of  the  instractional  system.  The  following 
sections  summarize  and  discuss  the  student  and  instructor  provided  feedback. 

Student  Feedback.  We  asked  the  students  to  assess  the  relevance  of  the  course 
content,  the  presentation  of  course  materials,  the  value  of  the  information  presented,  and 
the  extent  to  which  the  defined  course  objectives  were  met.  Table  4  provides  a  summary 
of  their  responses. 

As  a  method  of  summative  evaluation,  the  student  conments  provide  a  valuable 
resource  in  evaluating  the  IRM  course.  Summative  evaluation  provides  an  opportunity  to 
evaluate  an  instructional  system  in  whole.  In  as  much  as  this  administration  of  the  course 
was  its  first,  the  student  feedback  provides  an  initial  look  at  how  the  course  as  a  whole  can 
be  improved. 
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Table  4 


_ Summary  of  Student  Feedback  on  IRM  Course _ 

Measure  +  or-  Comments 

Relevance  +  Material  was  generally  enlightening  (i.e.  new  to  the  students) 

+  Provided  a  good  overview  of  IRM  processes 
Material  should  be  less  academically  oriented 
Need  to  make  a  clearer  tie  to  duty  performance 
Some  of  the  material  was  difficult  to  understand  (Modules  6  and  7) 
Course  needs  a  summary  module  to  tie  the  material  together 
Need  to  provide  more  military  examples 

Presentation  +  Slides  and  handouts  were  generally  appropriate 

+  Instructors  were  knowledgeable  and  professional 

Student  handouts  had  too  many  slides  per  page  (hard  to  read) 

Need  more  exercises  and  case  studies 
Slides  need  more  color  and  graphics 
Having  mote  than  one  instructor  was  distracting 

Value  +  Felt  more  knowledgeable  about  IRM  in  general 

+  Anxious  to  share  their  learning  in  the  workplace 
+  IRM  concepts  presented  in  course  are  important  enough  to  be 
included  in  skills  knowledge  tests  (SKT) 

Met  Objectives  +  Objectives,  as  stated  within  the  course,  were  met 

Did  not  feel  confident  in  the  depth  of  IRM  comprehension 
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Overall,  the  students  seemed  to  feel  that  the  content  of  the  course  was  relevant  to 
them  as  information  resources  managers.  Likewise,  their  responses  seem  to  indicate  that 
the  students  were  of  the  opinion  that  the  course  objectives  were  met.  In  addition,  all 
students  reported  feeling  as  though  the  course  had  provided  valuable  information  and 
knowledge. 

However,  the  students  were  critical  of  some  aspects  of  the  instructional  system. 

In  general,  the  students  felt  that  the  course  could  be  made  more  relevant  to  their  needs  by 
incorporating  more  military  examples  and  case  studies.  Likewise,  the  students  suggested 
an  effort  to  make  the  orientation  of  the  course  more  practical  and  less  academic.  Further, 
the  students  reported  that  the  delivery  of  the  course  would  be  better  if  improvements  were 
made  to  the  slides  and  handouts.  They  reported  that  the  slides  needed  more  color  and 
graphics. 

Instructor  Feedback.  We  also  asked  the  instructors  to  assess  the  relevance  of  the 
course  content,  the  presentation  of  course  materials,  and  the  extent  to  which  the  defined 
course  objectives  were  met.  Table  5  provides  a  summary  of  their  responses. 

Just  as  the  students’  feedback  provided  a  mechanism  for  summative  evaluation,  so 
too  did  the  instructors’  comments.  Without  exception,  the  instmctors  felt  that  the  course 
material  was  relevant  to  the  target  audience  (mid-level  information  resources  managers). 
Likewise,  the  three  instmctors  agreed  that  course  objectives  were  met.  However,  the 
instmctors  did  provide  suggestions  for  improving  the  course  presentation  and  for  making 
the  course  even  more  relevant.  Specifically,  the  instmctors  indicated  that  the  slides  and 
instractor  notes  lack  uniformity.  They  suggested  providing  all  sections  of  instmctor  notes 
and  slides  in  a  consistent  format.  Likewise,  they  thought  the  slides  should  be  incorporated 
into  the  notes  to  help  the  instmctor  associate  particular  slides  with  the  appropriate 
supporting  material  in  the  notes. 


27 


Table  5 


Summary  of  Instructor  Feedback  on  IRM  Course 


Measure 

+  or- 

Comments 

Relevance 

+ 

Provides  a  good  short  overview  of  the  disciplines  of  IRM 

+ 

Appropriate  content  for  mid-level  information  resources  managers 

Could  improve  relevance  with  more  specific  examples  and  case 

studies 

Presentation 

+ 

Course  material  was  generally  appropriate 

- 

Instructor  notes  could  be  more  uniform  in  format 

Instructor  notes  should  incorporate  the  slides  to  provide  better 

picture  of  what  is  to  be  presented 

Met  Objectives 

+ 

Course  objectives  were  met 

Analysis  and  Discussion  of  Quantitative  Data 

Mean  scores  for  the  pretest  and  posttest  were  calculated  for  each  of  the  seven 
modules.  Dependent  t  tests  were  computed  for  each  pretest-posttest  matched  pair  to 
determine  if  the  scores  on  the  pretest  were  significantly  lower  than  the  scores  on  the 
posttest.  The  alpha  level,  p,  of  .20  was  chosen  for  analyzing  the  individual  modules  due  to 
the  unusually  small  sample  size  with  which  we  were  working  and  because  we  had  a 
commensurate  need,  in  dealing  with  a  new  course,  to  improve  statistical  power.  Table  6 
shows  that  significant  differences  were  found  for  five  of  the  seven  modules. 
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In  all  cases,  the  posttest  scores  were  higher  than  the  pretest  scores,  as  evidenced 
by  the  percent  increase  column  of  Table  6.  This  would  seem  to  indicate,  at  least  for 
modules  1, 2,3,5,  and  6,  that  the  course  had  a  positive  effect  on  the  student's  test  scores. 
The  overall  test  scores  showed  a  positive  increase  of  34  percent  from  the  pretest  to  the 
posttest.  This  difference  was  shown  to  be  significant  at  an  alpha  level  less  than  .005. 
However,  this  increase  could  be  attributable  to  the  "testing"  effect,  since  they  were 
exposed  to  the  same  instrument  twice  within  a  period  of  three  days. 


Table  6 


PretestI Posttest  Scores:  Descriptive  Statistics  and  Tests  of  Significance  Between  Means 


Module 

%  Increase 

Mean 

Difference 

Standard 

Deviation 

t  Statistic 

p-value 

1* 

25.0 

-1.0000 

.5773 

-1.73 

.  .1438 

2* 

45.5 

-0.8333 

.5426 

-1.54 

.1852 

3* 

200.0 

-0.6666 

.2108 

-3.16 

.0250 

4 

50.0 

-1.3333 

.9189 

-1.45 

.2065 

5* 

18.2 

-1.3333 

.7601 

-1.75 

.1398 

6* 

42.3 

-1.8333 

.7923 

-2.31 

.0686 

7 

30.0 

-1.0000 

.8165 

-1.22 

.2752 

Total  Test** 

34.0 

-8.0000 

1.6329 

-4.90 

.0045 

^Significantly  different  ( p<,20) 
^^Significantly  different  (p<.005) 


Summary 


This  chapter  presented  an  analysis  and  discussion  of  the  qualitative  and 
quantitative  data  gathered  in  during  our  research.  The  qualitative  data,  derived  from 
surveys  of  the  students  and  instructors,  provided  a  mechanism  for  summatively  evaluating 
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the  IRM  course.  From  this  summative  evaluation,  we  were  able  to  identify  potential 
improvements  to  the  course.  The  quantitative  data,  gathered  using  the  measurement 
instrument  developed  to  evaluate  how  well  the  course  meets  its  objectives,  led  to  insight 
into  the  effectiveness  of  the  instmctional  system. 
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rv.  Conclusions  and  Recommendations 


Introduction 

Based  on  quantitative  and  qualitative  data  gathered  regarding  the  administration  of 
an  IRM  course,  we  have  drawn  conclusions  about  and  developed  recommendations  for  the 
course.  In  this  chapter  we  will  present  our  conclusions  regarding  course  effectiveness 
based  on  an  assessment  of  the  quantitative  and  qualitative  data  presented  in  Chapter  III. 
Afterwards,  we  will  discuss  the  limitations  of  our  research.  Finally,  we  will  discuss  areas 
in  need  of  further  research,  specifically  addressing  recommended  changes  to  the 
measurement  instmment  and  instmctional  system. 

Conclusions 

Quantitative  Assessment.  The  difference  in  scores  on  the  pretest  and  posttest 
seem  to  indicate  that  the  students  had  a  better  understanding  of  the  IRM  concepts 
presented  in  the  objectives  of  the  course  after  being  exposed  to  the  course  of  instruction. 
However,  many  causes  could  explain  reasons  for  the  change,  and  other  possibilities  would 
have  to  be  ruled  out  before  a  definitive  statement  about  course  effectiveness  could  be 
made.  Even  still,  the  significance  of  the  change  in  total  test  scores  does  give  confidence 
that  a  positive  change  in  relevant  knowledge  about  this  discipline  has  occurred.  In  other 
words,  although  the  effectiveness  of  particular  portions  of  the  course  may  be  in  question, 
as  a  whole,  the  course  is  effective. 

Qualitative  Assessment.  Both  instractors  and  students  indicated  that  the  course 
was  basically  sound  and  provided  a  useful  overview  of  IRM  concepts  and  principles  for 
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practicing  mid-level  information  resources  managers.  In  particular,  the  students  and 
instructors  indicated  that,  in  their  opinions,  the  course  objectives  had  been  met.  Based  on 
the  feedback  from  the  groups,  we  feel  confident  that  the  course  is  a  valuable  tool  for 
familiarizing  managers  with  IRM  concepts  and  principles.  However,  realizing  that  the 
students  and  instructors  did  provide  some  criticism  of  the  instmctional  system,  this 
version  of  the  course  may  not  be  appropriate  for  field  implementation  until  their  concerns 
are  addressed.  In  the  recommendations  section  of  this  chapter,  we  will  provide  specific 
suggestions  for  how  the  instructional  system  can  be  made  ready  for  operational 
deployment. 

Limitations  of  the  Research 

Due  to  the  nature  of  our  research  design,  many  factors  cause  uncertainty  regarding 
the  results  of  our  study.  Some  of  the  more  obvious  problems  we  faced  were  small  sample 
size,  extraneous  factors  possibly  affecting  validity,  possibly  inadequate  construction  of  the 
measurement  instrument,  and  low  reliability  estimates. 

Sample  Size.  We  were  only  able  to  find  six  students  who  matched  the  course's 
intended  audience  to  participate  in  this  study.  It  is  difficult  to  determine  whether  the  data 
gathered  from  such  a  small  group  can  provide  meaningful  information  about  the  course. 
Likewise,  the  small  sample  used  for  our  study  could  have  caused  problems  such  as  the  low 
reliability  estimates  discussed  later. 

Confounds.  It  is  possible  that  extraneous  factors  may  have  influenced  the  validity 
of  our  quantitative  assessment.  Specifically,  the  effect  of  testing  and  reactivity  could  not 
be  controlled  in  our  design.  The  students  may  have  scored  higher  on  the  posttest  simply 
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because  they  were  exposed  to  the  measurement  instrument  twice  within  a  period  of  three 
days  and  were  more  comfortable  with  the  posttest.  Another  explanation  is  that  the 
students  were  sensitized  to  the  material  covered  by  the  pretest  and  paid  particular 
attention  to  those  topics  as  they  were  presented  in  the  course.  While  sensitization  to 
course  material  by  the  pretest  may  have  caused  the  difference  in  test  scores  to  be  more 
pronounced,  the  very  idea  that  sensitization  could  have  influenced  scores  would  imply  the 
course  is  at  least  somewhat  effective. 

Instrument  Construction.  After  completing  our  data  collection,  we  discovered  that 
we  used  what  we  now  feel  was  an  inappropriate  criterion  for  selecting  items  to  be 
included  in  the  final  version  of  the  measurement  instrument.  Several  authors  suggest 
selecting  items  with  an  average  p-value  of  .5  for  a  maximally  discriminating  measurement 
instrument  (Guion,  1965:49;  Helmstadter,  1964:169).  Lord  asserts,  however,  that  for  a 
multiple-choice  item  with  five  alternatives,  an  average  p-value  of  .7  is  more  appropriate. 
He  argues  that  having  an  easier  test  reduces  the  effect  of  chance  guessing  and  increases 
reliability  (Lord,  1952:193-194). 

Reliability.  Of  the  five  modules  that  showed  a  significant  increase  in  scores  from 
the  pretest  to  the  posttest,  only  three  had  reliability  estimates  higher  than  .30.  While  it  is 
impossible  to  set  exact  standards  forjudging  reliability  estimates,  it  is  generally  accepted 
that  a  reliability  above  .80  is  satisfactory  forjudging  the  accuracy  of  individual  scores.  As 
a  general  rule,  much  lower  reliability  estimates  are  acceptable  forjudging  the  accuracy  of  a 
group  of  scores.  However,  with  a  reliability  estimate  of  .30,  we  are  approaching,  if  not 
already  past,  the  point  at  which  no  conclusions  can  be  drawn  about  the  accuracy  of  the 
scores  (Womer,  1968:41).  The  low  reliability  estimates  could  be  attributable  to  the 
instability  of  small-N  correlational  estimates. 
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Recommendations 


Several  areas  of  this  study  could  provide  the  basis  for  further  research.  The  same 
design  could  be  used  with  a  larger  group  of  students  to  determine  if  more  meaningful 
results  can  be  attained.  In  addition,  another  method  of  developing  the  measurement 
instrument  could  increase  confidence  in  the  accuracy  of  data  gathered  with  it.  Likewise, 
modifications  to  the  instructional  system  could  improve  the  overall  results  of  future 
studies.  Our  suggestions  for  improvements  to  the  measurement  instrument  and  the 
instructional  system  follow. 

Measurement  Instrument.  A  new  measurement  instrument  should  be  created  using 
the  criteria  laid  out  in  this  chapter.  We  were  able  to  develop  a  new  measurement 
instrument  using  the  original  pool  of  questions  surviving  the  validation  process.  We 
selected  items  using  Lord's  criterion  of  an  average  p-value  of  .7  and  Mehrens  and 
Lehmann's  criterion  of  item-discrimination  indices  of  at  least  .2.  The  item-discrimination 
index  can  be  computed  as  follows: 


Discrimination  = 


(3) 


where 


Ru  =  number  of  students  whose  scores  placed  them  above  the  50th  percentile  for  the  test 
and  who  answered  the  item  correctly 

/?£,  =  number  of  students  whose  scores  placed  them  below  the  50th  percentile  for  the  test 
and  who  answered  the  item  correctly 
T  =  total  number  of  students  tested  (Mehrens  and  Lehmann,  1991:162). 
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We  were  able  to  find  43  items  which  met  both  criteria  from  our  original  pool  of  questions. 
The  new  instrument  had  an  internal  consistency  estimate  of  .795  and  a  standard  error  of 
measurement  of  3.36,  which  is  considerably  better  than  the  original. 

It  is  possible  that,  with  minor  revisions  to  the  items  in  the  pool  and  subsequent 
reliability  testing,  a  longer  instrument  can  be  constructed  with  a  reliability  similar  to,  or 
better  than,  the  43  question  instrument  mentioned  in  the  previous  paragraph.  Our  review 
of  literature  has  revealed  that  examining  the  alternatives  within  each  item,  or  conducting 
distracter  analysis,  can  provide  insight  into  problems  with  the  item  (Mehrens  and 
Lehmann,  1991:163).  Due  to  time  constraints,  we  did  not  perform  distracter  analysis  on 
our  pool  of  questions. 

Instructional  System.  Our  recommendations  for  improving  the  course  are  centered 
on  upgrading  the  presentation  of  the  course  and  adjusting  the  focus  of  course  content. 

Course  Presentation.  For  the  purpose  of  this  study,  the  course  was  taught 
by  a  team  of  three  instructors.  A  major  complaint  from  the  students  involved  in  the  study 
was  that  there  was  some  confusion  brought  about  by  differences  in  each  instructor’s 
approach  to  the  material.  While  sharing  the  teaching  responsibilities  may  sometimes 
become  necessary  in  the  field,  for  the  purpose  of  consistency,  we  recommend  having  one 
instructor  teach  all  sections  of  the  course;  particularly  in  field  testing  the  final  version. 

Likewise,  we  recommend  that  an  effort  be  made  to  put  all  course  materials, 
including  slides,  instructor  notes  and  student  handouts,  into  a  more  standard  format.  In 
particular,  the  layout,  font  sizes,  and  style  of  writing  of  all  materials  needs  to  be 
standardized.  Both  students  and  instructors  commented  on  the  lack  of  standardization 
within  the  pilot  version  of  the  course.  Additionally,  the  incorporation  of  the  course  slides 
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into  the  instructor  notes  would  allow  instructors  to  more  effectively  prepare  for  course 
presentation. 


Course  Content.  Minor  revisions  to  course  content  could  enhance  the 
quality  of  the  course  greatly.  The  addition  of  case  studies,  illustrative  examples,  and 
classroom  exercises  to  the  course  should  allow  the  students  to  achieve  a  greater 
understanding  of  the  concepts  presented.  In  choosing  the  appropriate  material  for  these 
additions,  care  should  be  taken  to  ensure  the  course  is  made  more  relevant  for  the  people 
in  the  field.  The  pilot  test  feedback  from  the  students  criticised  some  of  the  current  course 
materials  for  being  “too  academic”  in  orientation.  We  suggest  changing  the  focus  of 
course  material  from  presentation  of  theory  and  knowledge  to  practical  application. 
Another  consideration  is  that  the  course  was  originally  intended  to  last  no  longer  than  one 
week.  We  recommend  additions  be  limited  to  meet  this  requirement  unless  it  can  be 
shown  that  the  benefit  of  lengthening  the  course  outweighs  the  added  cost. 
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Appendix  A:  Student  Feedback  Form 

Instructions:  Please  respond  to  the  following  questions  as  honestly  and  directly  as 
possible.  Your  comments  will  provide  valuable  feedback  which  will  be 
used  to  improve  the  content  and  presentation  of  this  course.  Remember, 
this  is  the  first  administration  of  the  course  material.  Consequently,  the 
course  developers  and  the  Air  Staff  are  relying  on  your  critique  to  help 
make  this  course  more  relevant  and  effective  for  future  students. 


How  did  you  feel  about  the  relevance  of  the  content  of  this  course?  (Be  as  specific  as 
possible.) 


How  did  you  feel  about  the  presentation  (slides,  instruction,  handouts)  of  this  course? 
How  could  the  presentation  be  improved? 


Do  you  feel  this  course  provided  you  with  valuable  information/knowledge?  Why  or  Why 
not? 


The  purpose  of  this  course  was  to  familiarize  you  with  basic  IRM  concepts  and  principles. 
To  what  extent  do  you  feel  this  objective  was  met? 


If  you  would  like  to  provide  additional  comments,  please  do  so  here. 


THANK  YOU  FOR  YOUR  FEEDBACK! 


Capt  Brenda  Forcht  and  Lt  Larry  Cox 


Appendix  B:  Instructor  Feedback  Form 


Instructions:  Please  respond  to  the  following  questions  as  honestly  and  directly  as 
possible.  Your  comments  will  provide  valuable  feedback  which  will  be 
used  to  improve  the  content  and  presentation  of  this  course.  Remember, 
this  is  the  first  administration  of  the  course  material.  Consequently,  the 
course  developers  and  the  Air  Staff  are  relying  on  your  critique  to  help 
make  this  course  more  relevant  and  effective  for  future  instructors  and 
students. 


How  did  you  feel  about  the  relevance  of  the  content  of  this  course?  (Be  as  specific  as 
possible.) 


How  did  you  feel  about  the  course  materials  (slides,  instructor  notes,  handouts)?  Were 
they  helpful?  How  could  the  course  materials  be  improved? 


The  purpose  of  this  course  is  to  familiarize  students  with  basic  IRM  concepts  and 
principles.  To  what  extent  do  you  feel  the  course  meets  this  objective? 


If  you  would  like  to  provide  additional  comments,  please  do  so  here. 


THANK  YOU  FOR  YOUR  FEEDBACK! 


Capt  Brenda  Forcht  and  Lt  Larry  Cox 
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1.  Which  of  the  following  is  NOT  a  cross  life-cycle  activity? 

**  1.  design 

2.  process  management 

3.  measurement 

4.  documentation 

5.  feasibility  analysis 

2.  Over  half  the  workers  in  the  U.S.  today  are: 

1.  directly  involved  in  the  manufacture  of  products. 

**  2.  knowledge  workers. 

3.  blue-collar  workers. 

4.  clerical  and  secretarial  workers. 

5.  factory  workers. 

3.  Which  of  the  following  best  describes  the  concept  of  horizontal  integration? 

1.  assuring  information  systems  support  business  processes 
**  2.  spanning  across  functional  boundaries 

3.  maintaining  consistency  throughout  all  phases  of  systems  development 

4.  making  various  systems  work  cooperatively 

5.  deriving  systems  from  information  requirements 

4.  Why  is  risk  assessment  an  important  part  of  strategic  planning? 

1.  Allows  the  organization  to  determine  the  level  of  risk  it  is  willing  to  accept 

2.  Allows  the  organization  to  evaluate  potential  impact  of  opportunities  & 
threats  in  the  environment 

3.  Allows  the  organization  to  evaluate  potential  impact  of  strengths  &  weaknesses 

4.  Allows  the  organization  to  estimate  the  impact  of  strategic  options 

**  5.  All  of  the  above 

5.  In  which  phase  of  functional  process  improvement  are  opportunities  identified  based  on 
an  analysis  of  the  AS-IS  baseline? 

1.  execution 

2.  definition 

**  3.  preparation 

4.  data  analysis 

5.  strategic  planning 
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6.  Which  of  the  following  is  the  graphical  representation  of  a  business  process  showing 
interactions  and  interdependencies  between  tasks  (shows  how  a  functional  process 
works)? 

1.  data  model 

2.  business  model 

**  3.  activity  model 

4.  information  model 

5.  interaction  model 

7.  Which  of  the  following  is  NOT  indicative  of  the  manual  processing  phase  of  IRM 
evolution? 

1.  Records  were  kept  on  paper. 

2.  Data  stores  were  physically  located  in  offices. 

3.  Similar  information  was  stored  in  multiple  locations. 

**  4.  Firms  had  integrated  databases. 

5.  Update,  storage,  and  retrieval  activities  were  performed  manually. 

8.  Which  of  the  following  estabhshed  the  General  Services  Administration? 

1.  Brooks  Act  of  1965 

2.  Federal  Records  Act  of  1968 

3.  Office  of  Management  and  Budget  (0MB)  Circular  A-130 

**  4.  Federal  Property  and  Administrative  Services  Act  of  1949 

5.  Computer  Security  Act  of  1987 

9.  What  are  critical  success  factors? 

1.  Skills  and  capabilities  that  are  present  in  the  organization 

2.  Estimates  of  future  development  and  impact 

3.  Foundation  to  allow  the  organization  to  achieve  its  vision 

**  4.  Areas  where  success  must  be  achieved  in  order  to  meet  organizational  goals 

5.  All  of  the  above 

10.  For  which  of  the  following  phases  of  the  systems  development  life-cycle  is  end-user 
training  and  documentation  an  output? 

1.  analysis 

2.  design 

**  3.  implementation 

4.  planning 

5.  support 
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11.  Which  of  the  foUowing  is  produced  by  or  results  from  an  activity? 

1.  input 

2.  control 

3.  entity 

4.  mechanism 

**  5.  output 

12.  The  task  of  observing  how  business  processes  are  carried  out  is  characteristic  of 
which  phase  of  the  systems  development  life-cycle? 

**  1.  analysis 

2.  design 

3.  implementation 

4.  planning 

5.  support 

13.  A  system 

1.  uses  its  inputs  economically  in  producing  its  outputs. 

2.  produces  outputs  that  best  meet  the  objectives  of  the  system. 

3.  provides  formal  informational  support  to  the  members  of  the  organization. 

4.  does  not  have  subsystems. 

**  5.  is  an  organized,  interacting,  interdependent,  and  integrated  set  of  components 

or  variables  that  work  together  to  achieve  a  common  objective. 

14.  The  advent  of  transaction  processing  systems  is  characteristic  of  which  of  the  phases 
of  IRM  evolution? 

1.  manual  processing  phase 
**  2.  computer  technology  phase 

3.  personal  computing  phase 

4.  IRM  phase 

5.  All  of  the  above 

15.  For  which  of  the  following  phases  of  the  systems  development  life-cycle  are  user- 
identified  systems  problems  a  primary  trigger? 

1.  analysis 

2.  design 

3.  implementation 

4.  planning 

**  5.  support 
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16.  Which  of  the  following  is  the  application  of  a  structured  methodology  to  define  a 
function's  “AS-IS”  environment,  to  include  objectives  and  supporting  strategies,  and  a 
program  of  incremental  improvements  made  through  functional,  technical,  and  economic 
analysis  and  decision  making. 

1.  business  area  analysis 

2.  value  chain  evaluation 

3.  critical  success  factor  formulation 

**  4.  functional  process  improvement 

5.  competitive  analysis 

17.  Which  of  the  following  establishes  an  individual's  right  to  request  information  from 
federal  agencies? 

1.  Privacy  Act  of  1974 

**  2.  Freedom  of  Information  Act  of  1966 

3.  Federal  Records  Act  of  1968 

4.  Computer  Security  Act  of  1987 

5.  Brooks  Act  of  1965 

18.  Which  of  the  following  is  a  primary  output  of  the  planning  phase  of  systems 
development  life-cycle? 

1.  business  requirements  statement 

2.  technical  design  statement 

3.  viable  information  system 

4.  existing  system  details  and  limitations 

**  5.  application  development  project 

19.  Which  of  the  following  factors  does  NOT  influence  an  organization's  environmental 
turbulence  (pace  of  change)? 

1.  technology 

2.  competition 

3.  end-user  computing 

**  4.  costs 

5.  environmental  complexity 
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20.  What  was  the  underlying  cause  of  the  evolution  of  IRM  concepts? 

1.  management  information  systems 

2.  economies  of  scale 

3.  automation  of  business  processes 

**  4.  technology  push  and  demand  pull 

5.  boundary  spanning 

21.  Which  of  the  following  is  NOT  characteristic  of  information? 

1.  Information  is  an  increment  of  knowledge. 

2.  Information  relies  on  the  context  and  general  knowledge  of  its  recipient  for  its 
significance. 

**  3.  Information  is  considered  a  raw  material. 

4.  Information  is  data  that  has  meaning. 

5.  Information  has  real  or  perceived  value. 

22.  A  thing  about  which  we  wish  to  keep  information  is  called  a(n) 

1.  data  relationship. 

2.  attribute. 

3.  subtype. 

**  4.  data  entity. 

5.  supertype. 

23.  Which  of  the  following  activities  is  best  described  as  ongoing  procedures  that 
establish  standards  for  activities,  methods,  tools,  and  deliverables  of  the  systems 
development  life-cycle? 

1.  fact-finding 

2.  documentation 

3.  estimation 

4.  measurement 

**  5.  process  management 

24.  Which  of  the  following  are  decomposition  diagrams  that  are  incomplete  or  that  violate 
the  mles  of  activity  analysis? 

1.  overview 

2.  context 

3.  primitive 

4.  systems 

**  5.  for  exposition  only 
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25.  Which  of  the  following  refers  to  vertical  integration? 

1.  assuring  information  systems  support  business  processes 

2.  spanning  across  functional  boundaries 

3.  maintaining  consistency  throughout  all  phases  of  systems  development 

**  4.  1  and  3  only 

5.  1,  2,  and  3 

26.  Which  of  the  following  has  NOT  evolved  as  a  goal  of  IRM? 

**  1.  Prove  the  feasibility  of  organizational  computing. 

2.  Maximize  the  benefit  of  information  resources  to  the  organization  as  a  whole. 

3.  Strategic  application  of  information  resources 

4.  Shared  information  resources  across  functions 

5.  Functional  management  responsibility  for  systems  requirements 

27.  Which  of  the  following  best  describes  the  relationship  between  data  and  information? 

**  1.  Data  is  the  raw  material  for  information. 

2.  Data  and  information  are  essentially  the  same. 

3.  Data  is  more  important  than  information. 

4.  Data  is  a  subset  of  information. 

5.  Data  has  no  relationship  to  information. 

28.  WTiy  must  systems  be  controlled? 

1.  systems  exhibit  differentiation 

2.  systems  have  inputs  and  outputs 

3.  components  of  a  system  interact 

**  4.  systems  exhibit  entropy 

5.  systems  are  goal-seeking 

29.  Which  of  the  following  is  the  primary  output  of  the  implementation  phase  of  systems 
development  life-cycle? 

1.  business  requirements  statement 

2.  technical  design  statement 

**  3.  viable  information  system 

4.  existing  system  details  and  limitations 

5.  application  development  project 
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30.  In  which  phase  of  information  engineering  are  organizational  mission  and  structure 
defined? 

**  1.  planning 

2.  analysis 

3.  design 

4.  construction 

5.  engineering 

31.  Which  of  the  following  applies  to  the  Paperwork  Reauthorization  Act  of  1986? 

1.  Required  development  and  revision  of  five-year  plans  to  meet  information 
technology  needs. 

2.  Provided  the  primary  set  of  regulations  for  governing  federal  management, 
acquisition,  and  use  of  information  technology 

3.  Required  maintenance  of  a  comprehensive  set  of  IRM  policies 

**  4.  1  and  3  only 

5.  1, 2,  and  3 

32.  Which  these  is  performed  during  the  analysis  phase  of  information  engineering? 

1.  identification  of  organizational  goals  and  objectives 
**  2.  development  of  functional  groups 

3.  translation  of  logical  models  into  physical  models 

4.  implementation  of  information  systems  projects 

5.  development  of  high-level  process  and  data  models 

33.  Which  of  these  is  performed  during  the  design  phase  of  information  engineering? 

1.  development  of  entity-relationship  diagrams 

2.  development  of  functional  groups 

**  3.  translation  of  logical  models  into  physical  models 

4.  implementation  of  information  systems  projects 

5.  development  of  high-level  process  and  data  models 

34.  Information  engineering  is  an  integrated  process.  Which  of  the  following  best 
describes  how  this  integration  is  accomplished? 

1.  organizational  goals  are  identified  in  a  bottom-up  fashion 

2.  systems  are  developed  to  support  information  requirements 

3.  information  requirements  are  derived  from  organizational  goals 

**  4.  2  and  3  only 

5.  1,  2,  and  3 
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35.  Which  of  the  following  refers  to  the  ongoing  study  of  a  problem  environment  to 
identify  problem-solving  possibilities? 

1.  analysis 
**  2.  planning 

3.  design 

4.  implementation 

5.  support 

36.  Good  data  management  is  a  tenet  of  IRM.  Which  of  the  following  is  NOT  an  element 
of  good  data  management? 

1.  data  elements  mean  the  same  thing  to  everyone  in  the  organization 
^  2.  data  elements  are  flexibly  defined  to  meet  situational  requirements 

3.  someone  must  be  responsible  for  the  data 

4.  everyone  who  needs  and  has  a  right  to  data  should  be  able  to  access  it 

5.  None  of  the  above  (All  are  elements  of  good  data  management.) 

37.  Which  of  the  following  are  potential  benefits  of  strategic  planning? 

1.  minimized  risk 

2.  improved  resource  utilization 

3.  reduced  redundancy 

4.  improved  flexibility 

**  5.  All  of  the  above 

38.  The  environment  consists  of  those  things  which  surround  and  affect  a  system.  Which 
of  the  following  could  be  part  of  a  systems  environment? 

1.  regulations 

2.  directives 

3.  customers 

4.  laws 

**  5.  All  of  the  above 

39.  Which  of  the  following  does  NOT  contribute  to  the  need  for  strategic  planning? 

1.  environmental  instability 

2.  desire  to  integrate  business  functions 

**  3.  predictable  future  events 

4.  rapidly  changing  technology 

5.  desire  to  identify  business  opportunities 
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40.  The  creeping  commitment  approach  to  systems  development  advocates  which  of  the 
following? 

1.  Only  incurred  costs  should  be  considered  when  reevaluating  project  feasibility. 

2.  Both  incurred  and  future  costs  should  be  considered  when  reevaluating  project 
feasibility. 

**  3.  Costs  that  have  already  been  incurred  are  irrelevant  in  the  project  continuation 

decision. 

4.  In  each  phase  of  the  systems  development  life-cycle,  systems  analysts  decide 
whether  or  not  additional  user  requirements  will  be  supported  by  the  new 
information  system. 

5.  All  costs  are  irrelevant  when  reevaluating  project  feasibility. 

41.  Information  must  be  available  when  needed.  This  statement  refers  to  which  of  the 
following  characteristics  of  information? 

1.  relevance 

2.  frequency  of  use 

3.  time  horizon 

**  4.  timeliness 

5.  organization 

42.  Which  of  the  following  activities  is  best  described  as  the  measurement  of  how 
beneficial  the  development  of  an  information  system  would  be  to  an  organization? 

**  1.  feasibility  analysis 

2.  documentation 

3.  estimation 

4.  measurement 

5.  process  management 

43.  Which  phase  of  information  engineering  features  automated  software  coding? 

1.  planning 

2.  analysis 

3.  design 

**  4.  constmction 

5.  engineering 


44.  Which  of  the  following  defines  the  future  direction  of  the  organization? 

**  1.  vision  statement 

2.  mission  statement 

3.  goals 

4.  objectives 

5.  strategies 

45.  Which  of  the  following  is  a  goal  of  CIM? 

**  1.  tie  DoD  organizations  together  through  common,  shared  data 

2.  place  increased  emphasis  on  information  technology  acquisition 

3.  implement  a  rigid  world- wide  information  infrastructure 

4.  minimize  duplication  through  increased  automation 

5.  All  of  the  above 

46.  Which  of  the  following  is  a  form  of  cost  accounting  that  focuses  on  the  costs  of 
performing  a  functional  process  rather  than  on  the  costs  associated  with  an  organizational 
unit? 

1.  job  order  costing 

2.  financial  accounting 

3.  process  costing 

**  4.  activity  based  costing 

5.  business  costing 

47.  Which  is  NOT  an  example  of  information? 

1.  The  Ajax  Company  owes  $4000  to  the  ABC  Company. 

2.  Mr.  Smith's  office  telephone  number  is  555-4567. 

3.  The  suggested  retail  price  of  a  Ford  Taurus  is  $17,000. 

**  4.  The  numbers  876,  315,  and  4704. 

5.  The  personnel  manager's  social  security  number  is  223-34-4987. 


48.  Which  of  the  following  is  true  about  the  relationship  between  the  formal  and  informal 
organizations? 

1.  the  formal  organization  has  more  rules  than  the  informal  organization 

2.  the  informal  organization  is  less  structured  than  the  formal  organization 

**  3.  the  informal  organization  is  the  social  interaction  within  the  structure  of  the 

formal  organization 

4.  1  and  2  only 

5.  1,  2,  and  3 
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49.  Which  of  the  following  is  most  likely  a  knowledge  worker? 

1.  factory  worker 

2.  construction  worker 

3.  data  entry  clerk 

4.  salesclerk 

**  5.  financial  consultant 

50.  Information  must  address  the  needs  of  the  user.  This  statement  refers  to  which  of  the 
following  characteristics  of  information? 

1.  timeliness 

**  2.  relevance 

3.  source 

4.  level  of  aggregation 

5.  time  horizon 

51.  In  which  of  the  phases  of  functional  process  improvement  are  the  action,  transition 
and  project  plans  put  into  motion  and  the  improvement  package  implemented? 

**  1.  execution 

2.  definition 

3.  preparation 

4.  data  analysis 

5.  strategic  planning 

52.  Which  of  the  following  is  NOT  an  example  of  a  mechanism? 

1.  machines 

2.  facilities 

**  3.  regulations 

4.  systems 

5.  personnel 

53.  Which  of  the  following  does  NOT  describe  the  duties  of  the  General  Accounting 
Office? 

1.  Independently  observes  government  programs 

2.  Audits  government  contractors  and  procuring  agencies 

3.  Issues  decisions  on  bid  protests 

**  4.  Monitors  compliance  with  government- wide  IRM  policies 

5.  Investigates  federal  ADP  policy,  use,  and  procurement 
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54.  Which  of  the  following  agencies  maintains  and  amends  the  procurement  and 
contracting  portions  of  the  Federal  Information  Resources  Management  Regulations 
(FIRMR)? 

**  1.  General  Services  Administration 

2.  General  Accounting  Office 

3.  Office  ofManagement  and  Budget 

4.  Defense  Information  Services  Agency 

5.  Office  of  Resource  Management 

55.  Environmental  analysis  does  NOT  include  which  of  the  following? 

1.  identification  of  threats. 

2.  identification  of  opportunities. 

3.  inventory  of  current  resources. 

**  4.  identification  of  major  products  and  services. 

5.  consideration  of  pending  legislation. 

56.  Which  of  the  following  is  NOT  a  step  in  performing  a  functional  economic  analysis? 

1.  statement  of  objective 

2.  sensitivity  analysis 

3.  cost/benefit  analysis 

**  4.  cost-volume-profit  analysis 

5.  identification  of  alternatives 

57.  Information  engineering  methodology  stresses  which  of  the  following? 

1.  customer  involvement 

2.  top-down  partitioning  of  models 

3.  quick  completion  of  systems 

4.  business  focus 

**  5.  All  of  the  above 

58.  The  rule  that  defines  the  maximum  number  of  occurrences  of  one  entity  for  a  single 
occurrence  of  the  related  entity  is  called 

1.  normalization. 

**  2.  cardinality. 

3.  entity  supertype. 

4.  data  analysis. 

5.  ordinality. 
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59.  Which  of  the  following  is  a  requirement  for  successful  functional  process 
improvement? 

1.  senior  leadership  support 

2.  organized  and  trained  project  teams 

3.  architectures  in  place 

4.  formalized  implementation  strategy  and  procedures 

**  5.  All  of  the  above 

60.  Which  of  the  following  shows  the  current  (baseline)  structure  of  a  functional  process? 

1.  data  model 

2.  implementation  model 

**  3.  AS-IS  model 

4.  essential  model 

5.  TO-BE  model 

61.  Which  of  the  following  involves  the  ability  to  break  a  large  business  problem  down 
into  its  component  parts,  analyze  the  various  aspects  of  the  problem,  and  then  assemble  a 
system  to  solve  the  problem? 

1.  general  business  knowledge 

2.  systems  analysis  and  design  skills 

3.  computer  programming  experience  and  expertise 

**  4.  problem-solving  skills 

5.  interpersonal  relations  skills 

62.  For  which  of  the  following  phases  of  the  systems  development  life-cycle  are  business 
and  information  system  plans  a  key  output? 

1.  analysis 

2.  design 

3.  implementation 

**  4.  planning 

5.  support 

63.  Which  of  the  following  agencies  is  the  key  monitor  of  federal  automated  data 
processing  equipment  (ADPE)  procurement? 

**  1.  General  Services  Administration 

2.  General  Accounting  Office 

3.  Officeof  Management  and  Budget 

4.  Federal  Contracting  Agency 

5.  Office  of  Resource  Management 
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64.  An  attribute  or  group  of  attributes  that  uniquely  identifies  one  and  only  one 
occurrence  of  an  entity  is  called  a(n) 

1.  data  relationship. 

2.  subtype. 

**  3.  key  attribute. 

4.  data  entity. 

5.  supertype. 

65.  Something  that  will  limit  flexibility  in  defining  a  solution  to  meet  objectives  is  called 
a(n) 

1.  mechanism. 

2.  directive. 

**  3.  control. 

4.  problem. 

5.  opportunity. 

66.  Which  of  the  following  is  NOT  a  characteristic  of  information  engineering 
methodology? 

1.  enterprise- wide  perspective 

2.  business  focus 

**  3.  bottom-up  identification  of  information  requirements 

4.  automated  systems  development 

5.  integrated  systems  development 

67.  Which  of  the  following  defines  the  scope,  mission,  business  objectives,  organizational 
structure,  duties  and  responsibilities,  and  management  strategy  for  process  improvement? 

1.  business  plan 

**  2.  functional  architecture 

3.  organization  chart 

4.  directives 

5.  None  of  the  above 

68.  What  does  a  strategic  plan  define? 

1.  products  and  services 

2.  mission 

3.  customers 

4.  values  and  beliefs 

**  5.  All  of  the  above 
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69.  Which  of  the  following  shows  the  objective  (target)  structure  of  a  functional  process? 

1.  data  model 

2.  implementation  model 

3.  AS-IS  model 

4.  essential  model 

**  5.  TO-BE  model 

70.  Which  of  the  following  is  the  primary  output  of  the  design  phase  of  systems 
development  life-cycle? 

1.  business  requirements  statement 
**  2.  technical  design  statement 

3.  viable  information  system 

4.  existing  system  details  and  limitations 

5.  application  development  project 

71.  The  Brooks  Act  gave  which  of  the  following  agencies  fiscal  and  policy  control  over 
procurement  and  management  of  ADPE? 

1.  General  Services  Administration 

2.  General  Accounting  Office 

**  3.  Office  of  Management  and  Budget 

4.  Defense  Information  Services  Agency 

5.  Internal  Revenue  Service 

72.  Which  of  the  following  would  be  accomplished  during  functional  economic  analysis? 

1.  identify  improvement  alternatives 

2.  identify  cost  of  improvement  alternatives 

3.  choose  among  improvement  alternatives 

4.  determine  the  consequences  of  past  decisions 

**  5.  All  of  the  above 

73.  Which  of  the  following  is  NOT  a  problem  associated  with  the  manual  processing 
phase  of  IRM  evolution? 

**  1.  Management  was  investing  heavily  in  information  technology. 

2.  Multiple  files  and  copies  had  to  be  maintained. 

3.  Users  had  to  physically  locate  the  data  store  containing  need  information. 

4.  Data  stores  and  reports  lacked  concurrency. 

5.  Creation  and  maintenance  of  data  stores  was  costly  and  time  consuming. 
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74.  Which  of  the  following  is  NOT  a  hallmark  of  an  information  society? 

1.  increased  turbulence 
**  2.  increased  number  of  industrial  jobs 

3.  dramatic  increase  of  available  knowledge 

4.  growth  of  complexity 

5.  All  of  the  above  are  hallmarks  of  an  information  society. 

75.  Which  of  the  following  disciplines  did  NOT  contribute  to  the  evolution  of  IRM? 

1.  Systems  Theory 

2.  Accounting 

3.  Cognitive  Science 

4.  Management  and  Organizational  Theory 

**  5.  None  of  the  above  (All  contributed  to  the  evolution  of  IRM) 

76.  Which  of  the  following  is  NOT  a  business  process  improvement  (BPI)  method? 

1.  activity  models 

2.  data  models 

3.  functional  economic  analysis 

**  4.  information  engineering 

5.  activity  based  costing 

77.  Which  of  the  following  is  NOT  true  of  information  engineering? 

1.  supports  the  entire  systems  development  life-cycle 

2.  seeks  to  integrate  systems  across  the  value  chain 

3.  is  data-centered  and  data-driven 

**  4.  is  designed  to  develop  software  for  individual  systems 

5.  seeks  to  streamline  organizational  procedures 

78.  What  is  a  source  of  demand  pull? 

1.  the  drive  to  automate  an  organization 

2.  innovations  in  information  technology 

3.  growing  complexity  of  social  organizations 

**  4.  needs  in  the  marketplace 

5.  evolution  of  IRM  concepts 
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79.  Which  of  the  following  is  the  primary  output  of  the  analysis  phase  of  systems 
development  life-cycle? 

**  1.  business  requirements  statement 

2.  technical  design  statement 

3.  viable  information  system 

4.  existing  system  details  and  limitations 

5.  application  development  project 

80.  Which  of  the  following  agencies  investigates  federal  automated  data  processing 
equipment  (ADPE)  policy,  use,  and  procurement? 

1.  General  Services  Administration 
**  2.  General  Accounting  Office 

3.  Office  of  Management  and  Budget 

4.  Federal  Contracting  Agency 

5.  All  of  the  above 

81.  In  which  phase  of  the  systems  development  life-cycle  are  information  systems 
applications  identified  and  prioritized? 

1.  analysis 

2.  design 

3.  implementation 

4.  support 

**  5.  planning 

82.  Which  of  the  following  diagram  types  shows  the  scope  of  an  activity  and  the  major 
inputs,  controls,  outputs,  and  mechanisms  in  an  activity? 

1.  overview 
**  2.  context 

3.  primitive 

4.  systems 

5.  for  exposition  only 

83.  Which  of  the  following  has  contributed  to  an  increase  in  available  knowledge? 

1.  increased  number  of  professional  and  trade  journals 

2.  increased  volume  of  private  and  public  sector  communications 

3.  increased  access  to  computerized  data  stores 

4.  Both  1  and  3  have  contributed. 

**  5.  All  of  the  above  have  contributed. 
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84.  Which  of  the  following  is  performed  during  the  construction  phase  of  information 
engineering? 

•  L  development  of  entity-relationship  diagrams 

2.  development  of  functional  groups 

3.  translation  of  logical  models  into  physical  models 

**  4.  implementation  of  information  systems  projects 

5.  development  of  high-level  process  and  data  models 

85.  Which  of  the  following  is  NOT  a  trigger  for  a  systems  development  project? 

1.  problems 

**  2.  feasibility  assessments 

3.  opportunities 

4.  directives 

5.  All  of  the  above 

86.  Which  of  the  following  is  NOT  an  element  of  a  strategic  plan? 

**  1.  operating  insoiictions  (01) 

2.  vision  statement 

3.  mission  statement 

4.  critical  success  factors 

5.  goals 

87.  Which  was  NOT  a  major  outcome  of  the  Paperwork  Reduction  Act  of  1980? 

1.  Formally  established  the  concept  of  IRM  within  the  federal  government 

2.  Made  each  agency  responsible  for  following  Office  of  Management  and  Budget 
(OMB)  information  policy  guidance 

3.  Assigned  central  information  management  responsibility  to  GSA 

**  4.  Required  federal  agencies  to  establish  multi-year  strategic  plans 

5.  Mandated  that  each  agency  designate  a  single  senior  official  to  carry  out 
responsibilities  under  the  act 

88.  After  investigating  the  issue  of  information  and  federal  agencies,  the  Congressional 
Committee  on  Federal  Paperwork  did  which  of  the  following? 

1.  Formally  established  the  concept  of  IRM  within  federal  government. 

2.  Created  the  Office  of  Management  and  Budget. 

3.  Assigned  central  information  management  responsibility  to  the  GS 1. 

4.  Provided  a  framework  for  managing  federal  information  resources. 

**  5.  Concluded  that  information  is  a  valuable  national  resource. 
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89.  Which  of  the  following  is  part  of  the  environmental  analysis  stage  of  strategic 
planning  process? 

**  1.  identification  of  internal  strengths  and  weaknesses 

2.  definition  of  organizational  mission 

3.  development  of  organizational  strategies 

4.  creation  of  organizational  vision  statement 

5.  All  of  the  above 

90.  In  which  of  the  phases  of  functional  process  improvement  are  objectives  and 
corresponding  strategies  established? 

1.  execution 
**  2.  definition 

3.  preparation 

4.  data  analysis 

5.  strategic  planning 

91.  Which  of  the  following  defines  how  the  organizational  mission  will  be  earned  out? 

1.  goals 

2.  objectives 

3.  strategies 

4.  critical  success  factors 

**  5.  1,  2,  and  3  only 

92.  Which  of  the  following  is  the  most  important  strategic  resource  of  the  Information 
Age? 

**  1.  information 

2.  labor 

3.  time 

4.  capital 

5.  money 

93.  The  task  of  obtaining  information  (or  fact-finding)  is  characteristic  of  which  phase  of 
the  systems  development  life-cycle? 

1.  analysis 

2.  design 

3.  implementation 

4.  planning 

**  5.  All  of  the  above 
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94.  What  is  benchmarking? 


1.  Identifying  those  things  that  must  go  right  for  the  mission  to  succeed 

2.  Identifying  the  current  level  of  productivity  within  the  organization 

**  3.  Identifying  comparative  measures  in  the  products  and  processes  of  the 

organization 

4.  Identifying  those  things  that  are  necessary  for  the  organization  to  stay 
competitive 

5.  Both  2  and  4 

95.  Raw  facts  in  isolation  that  are  not  useful  by  themselves  are  called 

**  1.  data. 

2.  attributes. 

3.  dataflow. 

4.  information. 

5.  entity. 

96.  Which  of  the  following  concepts  is  best  described  as  activities  that  overlap  many  or  all 
phases  of  the  systems  development  life-cycle? 

1.  analysis  activities 

2.  design  activities 

3.  implementation  activities 

4.  planning  activities 

**  5.  cross  life-cycle  activities 

97.  Which  of  the  following  is  NOT  part  of  the  Functional  Process  Improvement  Program 
vision? 

**  1.  multiple  interfaces 

2.  shared  information 

3.  functional  leadership 

4.  reusable  technology 

5.  mission  support 

98.  Which  of  the  following  is  performed  during  the  planning  phase  of  information 
engineering? 

1.  development  of  entity-relationship  diagrams 

2.  development  of  functional  groups 

3.  translation  of  logical  models  into  physical  models 

4.  implementation  of  information  systems  projects 

**  5.  development  of  high-level  process  and  data  models 
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99.  Which  of  the  following  participate  in  a  systems  development  life-cycle? 

1.  system  analysts 

2.  software  engineers 

3.  programmers 

4.  end-users 

**  5.  All  of  the  above 

100.  The  natural  association  that  exists  between  one  or  more  entities  is  called  a(n) 

1.  attribute. 

2.  supertype. 

**  3.  data  relationship. 

4.  identifier. 

5.  subtype. 

101.  Of  the  triggers  for  a  systems  development  project,  which  is  best  described  as  a 
chance  to  improve  the  organization? 

1.  problem 
**  2.  opportunity 

3.  feasibility  assessment 

4.  directive 

5.  None  of  the  above 

102.  Which  of  the  following  is  an  example  of  a  control? 

1.  machines 

2.  facilities 

**  3.  regulations 

4.  systems 

5.  personnel 
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103.  The  Federal  Information  Resources  Management  Regulations  (FIRMR)  does  NOT 
do  which  of  the  following? 

**  1.  Assign  General  Services  Administration  a  central  management  role  in 

government- wide  records  management. 

2.  Provide  the  primary  set  of  regulations  for  governing  federal  management, 
acquisition,  and  use  of  information  technology 

3.  Supplement  general  procurement  and  contracting  regulations  contained  in  the 
Federal  Acquisition  Regulations  (FAR) 

4.  Introduce  compliance  policies  pertaining  to  IRM  and  provisions  of  public  laws 

5.  None  of  the  above  (all  describe  the  FIRMR) 

104.  Which  of  the  following  is  a  business  case  decision  package  that  documents  and 
justifies  a  proposed  improvement  action? 

1.  financial  analysis 

2.  feasibility  assessment 

**  3.  functional  economic  analysis 

4.  value  chain  assessment 

5.  competitive  analysis 

105.  Which  of  the  following  systems  development  life-cycle  principles  implies  that  a 
systems  project  must  be  evaluated  for  cost  effectiveness? 

1.  Get  the  user  involved. 

2.  Establish  phases  and  activities. 

**  3.  Justify  systems  as  capital  investments. 

4.  Establish  standards  for  consistent  development  and  documentation. 

5.  Don't  be  afraid  to  cancel  or  revise  scope. 

106.  Which  of  the  following  is  data  or  material  that  is  transformed  in  an  activity? 

**  1.  input 

2.  control 

3.  entity 

4.  mechanism 

5.  output 
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107.  Which  of  the  following  is  NOT  a  CIM  implementation  principle? 

1.  Information  will  be  managed  through  centralized  control  and  decentralized 
execution. 

**  2.  Automation  is  preferred  to  simplification  through  elimination  and  integration. 

3.  Business  methods  should  be  routinely  scrutinized  by  cost-benefit  analysis  to 
include  benchmarking. 

4.  The  computer/communication  infrastructure  should  be  transparent  to  users. 

5.  Functional  management  is  responsible  for  benefits  and  costs  of  information 
systems. 

108.  Which  of  the  following  activities  is  best  described  as  recording  facts  and 
specifications  for  a  system? 

1.  fact-finding 
**  2.  documentation 

3.  estimation 

4.  measurement 

5.  process  management 

109.  Of  the  triggers  for  a  systems  development  project,  which  is  best  described  as  a 
requirement  that  is  imposed  by  management,  government,  or  some  external  influence? 

1.  problem 

2.  opportunity 

3.  feasibility  assessment 

**  4.  directive 

5.  threat 

1 10.  Which  of  the  following  activities  is  best  described  as  the  process  of  using  research, 
interviews,  meetings,  and  other  techniques  to  collect  information  about  systems, 
requirements,  and  preferences? 

**  1.  fact-finding 

2.  documentation 

3.  estimation 

4.  measurement 

5.  process  management 
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111.  Which  of  the  following  is  a  resource  that  is  used  to  turn  an  input  into  an  output? 

1.  input 

2.  control 

3.  entity 

**  4.  mechanism 

5.  output 

1 12.  Which  of  the  following  is  a  benefit  of  I-CASE  tool  use  in  information  engineering? 

**  1.  improved  identification  of  information  requirements 

2.  streamlined  business  processes 

3.  improved  business  focus 

4.  improved  customer  focus 

5.  All  of  the  above 

113.  Information  can  be  a  productivity  multiplier.  Which  of  the  following  does  NOT 
demonstrate  this  principle. 

**  1.  data  entry 

2.  coordinating  activities 

3.  boundary  spanning 

4.  decision  support 

5.  modeling 

1 14.  Which  of  the  following  is  a  CIM  strategy? 

**  1.  Develop  and  implement  cost  effective,  common  information  systems  based  on 

process  models  and  data  standards. 

2.  Develop  and  implement  a  computer/communication  infrastructure  based  on 
closed  system  principles. 

3.  Develop  data  standards  with  emphasis  on  departmental  segregation. 

4.  Manage  information  through  decentralized  control  and  centralized  execution. 

5.  All  of  the  above 
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115.  Business  process  improvement  principles  dictate  that  non- value  added  activities  be 
addressed  in  which  of  the  following  ways? 

1.  Non- value  added  activities  should  always  be  automated. 

2.  Non-value  added  activities  should  be  streamlined  and  simplified. 

**  3.  Non- value  added  activities  should  be  eliminated  when  not  mandated  by 

regulation  or  higher  authority. 

4.  Non-value  added  activities  should  be  modeled  but  not  streamlined. 

5.  Non- value  added  activities  should  not  be  considered  during  business  process 
improvement  efforts. 

1 16.  Which  of  the  following  constrains,  regulates,  directs,  and/or  focuses,  an  activity? 

1.  input 

**  2.  control 

3.  entity 

4.  mechanism 

5.  output 

117.  Which  of  the  following  activities  is  best  described  as  the  packaging  of 
documentation  for  review  by  interested  users  and  managers? 

1.  fact-finding 
**  2.  presentation 

3.  estimation 

4.  project  management 

5.  process  management 

118.  Which  of  the  following  systems  can  be  linked  to  the  evolution  of  IRM? 

1.  management  reporting  system 

2.  decision  support  system 

3.  executive  information  system 

4.  transaction  processing  system 

**  5.  All  of  the  above  can  be  linked  to  the  evolution  of  IRM. 

1 19.  Which  of  the  following  is  NOT  required  of  the  organization  in  response  to  the  shift 
to  an  information-based  society? 

**  1.  large  scale  investment  in  information  technology 

2.  continuous  product  and  process  innovation 

3.  constant  internal  organizational  renewal 

4.  protection  from  information  overload 

5.  organizational  design  for  knowledge  work  and  decision  making 
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120.  Why  is  IRM  strategic  planning  important  to  an  organization? 


**  1.  IRM  directly  supports  the  organizational  mission 

2.  All  functions  must  have  a  strategic  plan 

3.  IRM  strategic  plan  is  the  foundation  for  the  organizational  strategic  plan 

4.  IRM  strategic  plan  is  the  primary  input  for  acquisitions  planning 

5.  All  of  the  above 

121.  Distinct  activities  that  have  inputs  and  outputs  and  may  be  implemented  by  people, 
machines,  or  computers  are 

**  1.  business  processes. 

2.  computer  processes. 

3.  functions. 

4.  transactions. 

5.  dataflows. 

122.  A  data  modeling  tool  that  depicts  the  associations  among  different  categories  of  data 
within  a  business  is  called  a(n) 

1.  data  flow  diagram. 

2.  model. 

3.  implementation  model. 

4.  connectivity  diagram. 

**  5.  entity  relationship  diagram. 

123.  Facts  that  have  been  manipulated  so  they  are  useful  to  someone  are  called 

1.  data. 

2.  attributes. 

3.  dataflow. 

**  4.  information. 

5.  entity. 

124.  Which  of  the  following  provided  a  government- wide  policy  framework  for 
managing  federal  information  resources? 

1.  Brooks  Act 

2.  Federal  Records  Act 

**  3.  Office  of  Management  and  Budget  (0MB)  Circular  A- 130 

4.  Federal  Property  and  Administrative  Services  Act 

5.  Federal  Information  Resources  Management  Regulations  (FIRMR) 


125.  Which  of  the  following  is  NOT  a  primary  benefit  of  information  engineering  using  an 
I-CASE  tool? 

1.  quicker  systems  development 

2.  reduced  maintenance  costs 

3.  improved  software  portability 

4.  increased  mission  support 

**  5.  None  of  the  above 

126.  Which  of  the  following  is  inconsistent  with  CIM  principles? 

1.  Information  will  be  managed  through  centralized  control  and  decentralized 
execution. 

**  2.  Common  data  definitions  and  standards  should  be  developed  locally. 

3.  Systems  should  be  acquired  through  competitive  bidding  considering  internal 
and  external  sources. 

4.  Data  should  be  entered  only  once. 

5.  Interfaces  should  be  friendly  and  consistent. 

127.  What  key  event  sparked  the  need  for  the  development  of  IRM  methodologies? 

1.  the  discovery  of  new  information  storage  methods 

2.  the  application  of  computer  resources  to  information  processing  tasks 

3.  the  advent  of  organizational- wide  personal  computer  use 

**  4.  information  was  recognized  as  a  corporate  resource 

5.  All  of  the  above 

128.  The  basic  unit  in  activity  modeling  is  called  a(n) 

1.  entity. 

**  2.  ICOM. 

3.  attribute. 

4.  function. 

5.  dataflow. 

129.  Which  of  the  following  is  NOT  a  consequence  of  poor  planning? 

1.  redundancy 

**  2.  proactive  rather  than  reactive  management 

3.  suboptimization 

4.  1  and  3  only 

5.  1,  2,  and  3 
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130.  Which  of  the  following  is  NOT  a  basic  principle  of  systems  theory? 


1.  components  of  a  system  interact 
**  2.  systems  consist  of  uniform  components 

3.  systems  form  a  hierarchy 

4.  systems  exhibit  entropy 

5.  systems  transform  inputs  into  outputs 

131.  Which  of  the  following  is  NOT  a  strategy  of  the  CIM  program? 

1.  Develop  and  implement  cost  effective,  common  information  systems  based  on 
process  models  and  data  standards. 

2.  Manage  expenditures  for  information  regardless  of  the  technology  applied. 

3.  Apply  CIM  to  integrate  DoD-wide  operations. 

4.  Minimize  duplication  and  enhance  DoD  information  systems. 

**  5.  Enhance  DoD  information  systems  without  regard  to  cost  reduction  targets. 

132.  Which  of  the  following  is  NOT  true  of  an  IRM  strategic  plan? 

**  1.  It  is  formulated  to  evaluate  the  current  state  of  technology 

2.  It  is  derived  from  and  supports  the  organization's  strategic  plan 

3.  It  sets  the  broad  direction  and  goals  for  managing  information  within  the 
organization 

4.  It  presents  strategies  to  position  the  agency  to  take  full  advantage  of  the 
exploitation  of  technology 

5.  None  of  the  above  (all  are  true) 

133.  Which  of  the  following  is  the  graphical  representation  of  information  and  data  assets 
expressed  in  terms  of  entities  and  relationships? 

**  1.  data  model 

2.  business  model 

3.  activity  model 

4.  information  model 

5.  interaction  model 

134.  Which  of  the  following  laws  established  an  individual’s  right  to  review  federal 
records  containing  personal  information  about  him/her  maintained  by  federal  agencies? 

**  1.  Privacy  Act  of  1974 

2.  Freedom  of  Information  Act  of  1966 

3.  Federal  Records  Act  of  1968 

4.  Computer  Security  Act  of  1987 

5.  Brooks  Act  of  1965 


135.  Which  of  the  following  is  a  valuable  corporate  resource? 

1.  money 

2.  time 

3.  information 

4.  people 

**  5.  All  of  the  above  are  valuable  corporate  resources. 

136.  Which  of  the  following  agencies  exerts  general  supervisory  control  over  the 
budgetary  activities  of  executive  agencies? 

1.  General  Services  Administration 

2.  General  Accounting  Office 

**  3.  Office  of  Management  and  Budget 

4.  Defense  Information  Services  Agency 

5.  Internal  Revenue  Service 

137.  Which  of  the  following  is  NOT  an  important  component  of  functional  process 
improvement  efforts? 

1.  activity  modeling 
**  2.  information  engineering 

3.  benchmarking 

4.  activity  based  costing 

5.  data  modeling 

138.  Which  of  the  following  is  NOT  a  CIM  guiding  principle? 

1.  Customers  define  process  improvements  and  system  requirements,  manage 
information  and  measure  results. 

2.  Business  processes  must  be  simplified  and  improved  before  they  can  be 
automated. 

**  3.  Information  will  be  managed  through  decentralized  control  and  centralized 

execution. 

4.  The  fastest  progress,  at  lowest  risk,  is  achieved  by  evolution  not  revolution. 

5.  None  of  the  above 
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139.  Which  is  NOT  a  principle  of  systems  development? 

1.  Get  the  user  involved. 

2.  Establish  phases  and  activities. 

3.  Establish  standards  for  consistent  development  and  documentation. 

**  4.  Develop  systems  in  a  sequential  manner. 

5.  Don't  be  afraid  to  cancel  or  revise  scope. 

140.  Which  of  the  following  are  appropriate  responses  to  the  changes  brought  about  by 
the  transition  to  an  information  society? 

1.  protection  from  information  overload  and  decreased  flexibility 
**  2.  continuous  product  and  process  innovation 

3.  increased  structure  and  decreased  flexibility 

4.  implicit  mecharusms  for  acquisition  of  information  systems 

5.  organizational  design  for  office  automation 

141.  Which  of  the  following  systems  development  life-cycle  principles  reveals  strengths 
and  weaknesses  of  the  system  before  the  system  is  built,  stimulates  user  involvement,  and 
reassures  management  about  progress? 

1,  Get  user  involved. 

2.  Establish  phases  and  activities. 

**  3.  Establish  standards  for  consistent  development  and  documentation. 

4.  Use  a  problem-solving  approach. 

5.  Don't  be  afraid  to  cancel  or  revise  scope. 

142.  The  process  followed  in  developing  a  new  information  system  for  an  organization  is 
referred  to  as  which  of  the  following? 

1.  systems  analysis  and  design 

2.  program  development  life-cycle 

**  3.  systems  development  life-cycle 

4.  problemsolving 

5.  business  process  reengineering 

143.  What  defines  how  the  mission  will  be  accomplished  in  accordance  with 
organizational  values  and  beliefs? 

**  1.  vision  statement 

2.  mission  statement 

3.  goals 

4.  objectives 

5.  strategies 
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144.  Ad  hoc  computing  was  a  major  problem  of  which  of  the  phases  of  IRM  evolution? 

1.  manual  processing  phase 

2.  computer  technology  phase 

**  3.  personal  computing  phase 

4.  IRM  phase 

5.  All  of  the  above 

145.  Which  of  the  following  was  the  most  important  strategic  resource  of  the  Industrial 
Age? 


1.  information 

2.  labor 

3.  time 

**  4.  capital 

5.  money 

146.  Information  has  all  of  the  following  special  qualities  EXCEPT  that  it 

1.  is  not  consumed. 

**  2.  does  not  age. 

3.  must  be  provided  in  a  timely  manner. 

4.  loses  its  value  over  time. 

5.  must  be  managed. 

147.  What  is  the  source  of  technology  push? 

1.  the  drive  to  automate  an  organization 

**  2.  innovations  in  information  technology 

3.  growing  complexity  of  social  organizations 

4.  needs  in  the  marketplace 

5.  evolution  of  IRM  concepts 

148.  Characteristics  that  are  common  to  all  or  most  instances  of  a  particular  entity  are 
called 

**  1.  data  attributes. 

2.  identifiers. 

3.  data  relationships. 

4.  subtypes. 

5.  associative  entities. 
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149.  In  which  of  the  following  phases  are  software  packages  installed? 

1.  planning 

2.  analysis 

3.  support 

**  4.  implementation 

5.  design 

150.  What  primary  role  does  functional  economic  analysis  (FEA)  play  in  business  process 
improvement  efforts? 

1.  The  FEA  determines  the  correct  alternative  solution. 

2.  The  FEA  details  how  solution  alternatives  compare  to  competitors'  best 
business  practices. 

3.  The  FEA  provides  data  about  the  cost  of  performing  the  activities  that  are  part 
of  the  process  being  modeled. 

**  4.  The  FEA  provides  a  format  for  presenting  solution  alternatives  for  management 

consideration. 

5.  The  FEA  does  all  of  the  above. 

151.  Of  the  triggers  for  a  systems  development  project,  which  is  best  described  as 

an  undesirable  situation  that  prevents  an  organization  from  achieving  its  purpose,  goals, 
and  objectives? 

**  1.  problem 

2.  opportunity 

3.  feasibility  assessment 

4.  directive 

5.  policy 

152.  Which  of  the  following  Congressional  initiatives  officially  introduced  information 
resources  management  (IRM)  to  federal  agencies? 

1.  Privacy  Act  of  1974 
**  2.  Paperwork  Reduction  Act  of  1980 

3.  Brooks  Act  of  1965 

4.  Fairness  in  Contracting  Act 

5.  Government  Performance  and  Results  Act  of  1993 
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153.  Which  of  the  following  is  true  of  an  organization? 

1.  An  organization  is  a  group  of  individuals. 

2.  An  organization  is  rationally  coordinated. 

3.  An  organization's  members  work  toward  a  common  goal. 

4.  An  organization's  members  often  conform  to  formal  and  informal  standards. 

**  5.  All  of  the  above 

154.  Which  of  the  following  is  NOT  a  key  element  of  business  process  improvement 
(BPI)? 

1.  identifying  and  eliminating  non-value  added  activities 

2.  emphasizing  reuse  of  assets 

3.  automating  only  after  underlying  process  have  been  improved 

4.  integrating  processes,  physical  assets,  organizations,  and  data 

**  5.  simplifying  and  streamlining  non- value  added  activities 

155.  Which  of  the  following  is  the  objective  of  the  CIM  program? 

**  1.  Increase  military  effectiveness 

2.  Create  models  of  all  DoD  processes 

3.  Use  information  technology  to  achieve  cost  reductions  and  improve  efficiency 

4.  Develop  a  common  methodology  for  systems  development 

5.  Provide  justification  for  information  technology  investments 

156.  Which  of  the  following  factors  is  likely  to  contribute  to  the  increasing  complexity  of 
organizational  decision-making? 

1.  politics 

2.  economics 

3.  regulation 

4.  culture 

**  5.  All  of  the  above 

157.  Which  of  the  following  evaluation  criteria  is  concerned  with  tangible  and  intangible 
benefits  of  a  systems  solution? 

**  1.  economic  feasibility 

2.  operational  feasibility 

3.  technical  feasibility 

4.  schedule  feasibility 

5.  political  feasibility 
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158.  In  which  phase  of  the  systems  development  life-cycle  is  a  computer-based  technical 
solution  devised? 

1.  analysis 
**  2.  design 

3.  implementation 

4.  planning 

5.  support 

159.  Which  of  the  following  defines  the  purpose  of  the  organization? 

1.  vision  statement 
**  2.  mission  statement 

3.  goals 

4.  objectives 

5.  strategies 

160.  According  to  the  Paperwork  Reduction  Act,  the  director  of  the  Office  of 
Management  and  Budget  (0MB)  has  authority  to  do  which  of  the  following? 

1.  Develop  and  implement  government-wide  IRM  policies 

2.  Oversee  the  development  and  use  of  information  management  principles, 
standards,  and  guidelines 

3.  Evaluate  agency  information  management  practices 

4.  Monitor  agency  compliance  with  Office  of  Management  and  Budget  (0MB) 
information  management  policies 

**  5.  All  of  the  above 
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Appendix  D:  60-Item  Measurement  Instrument  (Pretest/Posttestl 

1.  Being  available  when  needed  refers  to  which  of  the  characteristics  of  information? 

1.  relevance 

2.  frequency  of  use 

3.  time  horizon 

**  4.  timeliness 

5.  organization 

2.  Which  of  the  following  is  a  business  case  decision  package  that  documents  and  justifies 
a  proposed  improvement  action? 

1.  financial  analysis 

2.  feasibility  assessment 

**  3.  functional  economic  analysis 

4.  value  chain  assessment 

5.  competitive  analysis 

3.  Which  of  the  following  activities  is  best  described  as  the  measurement  of  how 
beneficial  the  development  of  an  information  system  would  be  to  an  organization? 

**  1.  feasibility  analysis 

2.  documentation 

3.  estimation 

4.  measurement 

5.  process  management 

4.  A  thing  about  which  we  wish  to  keep  information  is  called  a(n) 

1.  data  relationship. 

2.  attribute. 

3.  subtype. 

**  4.  data  entity. 

5.  supertype. 

5.  Which  of  the  following  best  describes  the  concept  of  horizontal  integration? 

1.  assiuing  information  systems  support  business  processes 
**  2.  spanning  across  functional  boundaries 

3.  maintaining  consistency  throughout  all  phases  of  systems  development 

4.  making  various  systems  work  cooperatively 

5.  deriving  systems  from  information  requirements 
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6.  Something  that  will  limit  flexibility  in  defining  a  solution  to  meet  objectives  is  called 
a(n) 

1.  mechanism. 

2.  directive. 

**  3.  control 

4.  problem. 

5.  opportunity. 

7.  The  task  of  obtaining  information  (or  fact-finding)  is  characteristic  of  which  phase  of 
the  systems  development  life-cycle? 

1.  analysis 

2.  design 

3.  implementation 

4.  planning 

**  5.  All  of  the  above 

8.  Information  engineering  is  an  integrated  process.  Which  of  the  following  best 
describes  how  this  integration  is  accomplished? 

1.  organizational  goals  are  identified  in  a  bottom-up  fashion 

2.  systems  are  developed  to  support  information  requirements 

3.  information  requirements  are  derived  from  organizational  goals 

**  4.  2  and  3  only 

5.  1,2,  and  3 

9.  For  which  of  the  following  phases  of  the  systems  development  life-cycle  is  end-user 
training  and  documentation  an  output? 

1.  analysis 

2.  design 

**  3.  implementation 

4.  planning 

5.  support 
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10.  Which  of  the  following  activities  is  best  described  as  ongoing  procedures  that 
establish  standards  for  activities,  methods,  tools,  and  deliverables  of  the  systems 
development  life-cycle? 

1.  fact-finding 

2.  documentation 

3.  estimation 

4.  measurement 

**  5.  process  management 

11.  Which  of  the  following  evaluation  criteria  is  concerned  with  tangible  and  intangible 
benefits  of  a  systems  solution? 

**  1.  economic  feasibility 

2.  operational  feasibility 

3.  technical  feasibility 

4.  schedule  feasibility 

5.  political  feasibility 

12.  Which  of  the  following  refers  to  vertical  integration? 

1.  assuring  information  systems  support  business  processes  ■ 

2.  spanning  across  functional  boundaries 

3.  maintaining  consistency  throughout  all  phases  of  systems  development 

**  4.  1  and  3  only 

5.  1,  2,  and  3 

13.  A  data  modeling  tool  that  depicts  the  associations  among  different  categories  of  data 
within  a  business  is  called  a(n) 

1.  data  flow  diagram. 

2.  model. 

3.  implementation  model. 

4.  connectivity  diagram. 

**  5.  entity  relationship  diagram. 

14.  Which  of  the  following  diagram  types  shows  the  scope  of  an  activity  and  the  major 
inputs,  controls,  outputs,  and  mechanisms  in  an  activity? 

1.  overview 
**  2.  context 

3.  primitive 

4.  systems 

5.  for  exposition  only 
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15.  Which  of  the  following  is  true  about  the  relationship  between  the  formal  and  informal 
organizations? 

1.  the  formal  organization  has  more  mles  than  the  informal  organization 

2.  the  informal  organization  is  less  structured  than  the  formal  organization 

**  3.  the  informal  organization  is  the  social  interaction  within  the  structure  of  the 

formal  organization 

4.  1  and  2  only 

5.  1, 2,  and  3 

16.  In  which  phase  of  the  systems  development  life-cycle  are  information  systems 
applications  identified  and  prioritized? 

1.  analysis 

2.  design 

3.  implementation 

4.  support 

**  5.  planning 

17.  Which  of  the  following  is  NOT  part  of  the  Functional  Process  Improvement  Program 
vision? 

**  1.  multiple  interfaces 

2.  shared  information 

3.  functional  leadership 

4.  reusable  technology 

5.  mission  support 

18.  Which  of  the  following  Congressional  initiatives  officially  introduced  information 
resources  management  (IRM)  to  federal  agencies? 

1.  Privacy  Act  of  1974 
**  2.  Paperwork  Reduction  Act  of  1980 

3.  Brooks  Act  of  1965 

4.  Fairness  in  Contracting  Act 

5.  Government  Performance  and  Results  Act  of  1993 
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19.  Which  of  the  following  is  the  application  of  a  structured  methodology  to  define  a 
function's  “AS-IS”  environment,  to  include  objectives  and  supporting  strategies,  and  a 
program  of  incremental  improvements  made  through  functional,  technical,  and  economic 
analysis  and  decision  making? 

1.  business  area  analysis 

2.  value  chain  evaluation 

3.  critical  success  factor  formulation 

**  4.  functional  process  improvement 

5.  competitive  analysis 

20.  Which  of  the  following  are  appropriate  responses  to  the  changes  brought  about  by  the 
transition  to  an  information  society? 

1.  protection  from  information  overload  and  decreased  flexibility 
**  2.  continuous  product  and  process  innovation 

3.  increased  stracture  and  decreased  flexibility 

4.  implicit  mechanisms  for  acquisition  of  infomiation  systems 

5.  organizational  design  for  office  automation 

21.  Environmental  analysis  does  NOT  include  which  of  the  following? 

1.  identification  of  threats. 

2.  identification  of  opportunities. 

3.  inventory  of  current  resources. 

**  4.  identification  of  major  products  and  services. 

5.  consideration  of  pending  legislation. 

22.  Which  of  the  following  agencies  is  the  key  monitor  of  federal  automated  data 
processing  equipment  (ADPE)  procurement? 

**  1.  General  Services  Administration 

2.  General  Accounting  Office 

3.  Office  of  Management  and  Budget 

4.  Federal  Contracting  Agency 

5.  Office  of  Resource  Management 
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23.  The  rule  that  defines  the  maximum  number  of  occurrences  of  one  entity  for  a  single 
occurrence  of  the  related  entity  is  called 

1.  normalization. 

**  2.  cardinality. 

3.  entity  supertype. 

4.  data  analysis. 

5.  ordinality. 

24.  Which  of  the  following  would  be  accomplished  during  functional  economic  analysis? 

1.  identify  improvement  alternatives 

2.  identify  cost  of  improvement  alternatives 

3.  choose  among  improvement  alternatives 

4.  determine  the  consequences  of  past  decisions 

**  5.  All  of  the  above 

25.  Characteristics  that  are  common  to  all  or  most  instances  of  a  particular  entity  are 
called 

**  1.  data  attributes. 

2.  identifiers. 

3.  data  relationships. 

4.  subtypes. 

5.  associative  entities. 

26.  Which  of  the  following  is  a  form  of  cost  accounting  that  focuses  on  the  costs  of 
performing  a  functional  process  rather  than  on  the  costs  associated  with  an  organizational 
unit? 

1.  job  order  costing 

2.  financial  accounting 

3.  process  costing 

**  4.  activity  based  costing 

5.  business  costing 

27.  Which  is  NOT  a  principle  of  systems  development? 

1.  Get  the  user  involved. 

2.  Establish  phases  and  activities. 

3.  Establish  standards  for  consistent  development  and  documentation. 

**  4.  Develop  systems  in  a  sequential  manner. 

5.  Don’t  be  afi'aid  to  cancel  or  revise  scope. 
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28.  Information  has  all  of  the  following  special  qualities  EXCEPT  that  it 

1.  is  not  consumed. 

**  2.  does  not  age. 

3.  must  be  provided  in  a  timely  manner. 

4.  loses  its  value  over  time. 

5.  must  be  managed. 

29.  Which  of  the  following  is  NOT  an  important  component  of  functional  process 
improvement  efforts? 

1.  activity  modeling 

**  2.  information  engineering 

3.  benchmarking 

4.  activity  based  costing 

5.  data  modeling 

30.  Which  of  the  following  is  NOT  true  of  an  IRM  strategic  plan? 

**  1.  It  is  formulated  to  evaluate  the  current  state  of  technology 

2.  It  is  derived  from  and  supports  the  organization's  strategic  plan 

3.  It  sets  the  broad  direction  and  goals  for  managing  information  within  the 
organization 

4.  It  presents  strategies  to  position  the  agency  to  take  full  advantage  of  the 
exploitation  of  technology 

5.  None  of  the  above  (aU  are  true) 

31.  Which  of  the  following  are  decomposition  diagrams  that  are  incomplete  or  that  violate 
the  rules  of  activity  analysis? 

1.  overview 

2.  context 

3.  primitive 

4.  systems 

5.  for  exposition  only 

Ad  hoc  computing  was  a  major  problem  of  which  of  the  phases  of  IRM  evolution? 

1.  manual  processing  phase 

2.  computer  technology  phase 

3.  personal  computing  phase 

4.  IRM  phase 

5.  All  of  the  above 


32. 
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33.  Which  of  the  following  is  a  benefit  of  I-CASE  tool  use  in  information  engineering? 

**  1.  improved  identification  of  information  requirements 

2.  streamlined  business  processes 

3.  improved  business  focus 

4.  improved  customer  focus 

5.  All  of  the  above 

34.  Which  of  the  following  agencies  exerts  general  supervisory  control  over  the  budgetary 
activities  of  executive  agencies? 

1.  General  Services  Administration 

2.  General  Accounting  Office 

**  3.  Office  of  Management  and  Budget 

4.  Defense  Information  Services  Agency 

5.  Internal  Revenue  Service 

35.  Distinct  activities  that  have  inputs  and  outputs  and  may  be  implemented  by  people, 
machines,  or  computers  are 

**  1.  business  processes. 

2.  computer  processes. 

3.  functions. 

4.  transactions. 

5.  dataflows. 

36.  For  which  of  the  following  phases  of  the  systems  development  life-cycle  are  user- 
identified  systems  problems  a  primary  trigger? 

1.  analysis 

2.  design 

3.  implementation 

4.  planning 

**  5.  support 

37.  What  was  the  underlying  cause  of  the  evolution  of  IRM  concepts? 

1.  management  information  systems 

2.  economies  of  scale 

3.  automation  of  business  processes 

**  4.  technology  push  and  demand  pull 

5.  boundary  spanning 
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38.  In  which  phase  of  information  engineering  are  organizational  mission  and  stracture 
defined? 

**  1.  planning 

2.  analysis 

3.  design 

4.  construction 

5.  engineering 

39.  Which  of  the  following  laws  established  an  individual's  right  to  review  federal  records 
containing  personal  information  about  him/her  maintained  by  federal  agencies? 

**  1.  Privacy  Act  of  1974 

2.  Freedom  of  Information  Act  of  1966 

3.  Federal  Records  Act  of  1968 

4.  Computer  Security  Act  of  1987 

5.  Brooks  Act  of  1965 

40.  Which  of  the  following  is  NOT  a  business  process  improvement  (BPI)  method? 

1.  activity  models 

2.  data  models 

3.  functional  economic  analysis 

**  4.  information  engineering 

5.  activity  based  costing 

41.  Which  of  the  following  has  NOT  evolved  as  a  goal  of  IRM? 

**  1.  Prove  the  feasibility  of  organizational  computing. 

2.  Maximize  the  benefit  of  information  resources  to  the  organization  as  a  whole. 

3.  Strategic  application  of  infonnation  resources 

4.  Shared  information  resources  across  functions 

5.  Functional  management  responsibility  for  systems  requirements 

42.  Which  of  the  following  is  NOT  a  characteristic  of  information  engineering 
methodology? 

1.  enterprise- wide  perspective 

2.  business  focus 

**  3.  bottom-up  identification  of  information  requirements 

4.  automated  systems  development 

5.  integrated  systems  development 
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43.  Which  of  the  following  defines  the  future  direction  of  the  organization? 

**  1.  vision  statement 

2.  mission  statement 

3.  goals 

4.  objectives 

5.  strategies 

44.  Which  of  the  following  is  the  graphical  representation  of  information  and  data  assets 
expressed  in  terms  of  entities  and  relationships? 

**  1.  data  model 

2.  business  model 

3.  activity  model 

4.  information  model 

5.  interaction  model 


45.  Which  of  the  following  involves  the  ability  to  break  a  large  business  problem  down 
into  its  component  parts,  analyze  the  various  aspects  of  the  problem,  and  then  assemble  a 
system  to  solve  the  problem? 

1.  general  business  knowledge 

2.  systems  analysis  and  design  skills 

3.  computer  programming  experience  and  expertise 

**  4.  problem-solving  skills 

5.  interpersonal  relations  skills 

46.  Which  of  the  following  is  NOT  required  of  the  organization  in  response  to  the  shift  to 
an  information-based  society? 

**  1.  large  scale  investment  in  information  technology 

2.  continuous  product  and  process  innovation 

3.  constant  internal  organizational  renewal 

4.  protection  from  information  overload 

5.  organizational  design  for  knowledge  work  and  decision  making 

47.  Which  of  the  following  is  part  of  the  environmental  analysis  stage  of  sttategic 
planning  process? 

**  1.  identification  of  internal  strengths  and  weaknesses 

2.  definition  of  organizational  mission 

3.  development  of  organizational  strategies 

4.  creation  of  organizational  vision  statement 

5.  All  of  the  above 
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48.  Which  of  the  following  is  NOT  true  of  information  engineering? 

1.  supports  the  entire  systems  development  life-cycle 

2.  seeks  to  integrate  systems  across  the  value  chain 

3.  is  data-centered  and  data-driven 

**  4.  is  designed  to  develop  software  for  individual  systems 

5.  seeks  to  streamline  organizational  procedures 

49.  Information  engineering  methodology  stresses  which  of  the  following? 

1.  customer  involvement 

2.  top-down  partitioning  of  models 

3.  quick  completion  of  systems 

4.  business  focus 

**  5.  All  of  the  above 

50.  Why  must  systems  be  controlled? 

1.  systems  exhibit  differentiation 

2.  systems  have  inputs  and  outputs 

3.  components  of  a  system  interact 

**  4.  systems  exhibit  entropy 

5.  systems  are  goal-seeking 

51.  Good  data  management  is  a  tenet  of  IRM.  Which  of  the  following  is  NOT  an  element 
of  good  data  management? 

1.  data  elements  mean  the  same  thing  to  everyone  in  the  organization 
**  2.  data  elements  are  flexibly  defined  to  meet  situational  requirements 

3.  someone  must  be  responsible  for  the  data 

4.  everyone  who  needs  and  has  a  right  to  data  should  be  able  to  access  it 

5.  None  of  the  above  (All  are  elements  of  good  data  management.) 

52.  Over  half  the  workers  in  the  U.S.  today  are 

1.  directly  involved  in  the  manufacture  of  products. 

**  2.  knowledge  workers. 

3.  blue-collar  workers. 

4.  clerical  and  secretarial  workers. 

5.  factory  workers. 


83 


53.  For  which  of  the  following  phases  of  the  systems  development  life-cycle  are  business 
and  information  system  plans  a  key  output? 


1.  analysis 

2.  design 

3.  implementation 

4.  planning 

5.  support 


54.  Which  of  the  following  is  NOT  a  trigger  for  a  systems  development  project? 

1.  problems 

**  2.  feasibility  assessments 

3.  opportunities 

4.  directives 

5.  All  of  the  above 

55.  Which  of  the  following  is  NOT  an  example  of  a  mechanism? 

1.  machines 

2.  facilities 

**  3.  regulations 

4.  systems 

5.  personnel 

56.  Which  of  the  following  factors  does  NOT  influence  an  organization's  environmental 
turbulence  (pace  of  change)? 

1.  technology 

2.  competition 

3.  end-user  computing 

**  4.  costs 

5.  environmental  complexity 

57.  Which  of  the  following  activities  is  best  described  as  recording  facts  and  specifications 
for  a  system? 


1.  fact-finding 
**  2.  documentation 

3.  estimation 

4.  measurement 

5.  process  management 
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58.  Which  of  the  following  is  a  goal  of  CIM? 

**  1.  tie  DoD  organizations  together  through  common,  shared  data 

2.  place  increased  emphasis  on  information  technology  acquisition 

3.  implement  a  rigid  world-wide  information  infrastructure 

4.  minimize  duplication  through  increased  automation 

5.  All  of  the  above 

59.  Which  of  the  following  does  NOT  contribute  to  the  need  for  strategic  planning? 

1.  environmental  instability 

2.  desire  to  integrate  business  functions 

**  3.  predictable  future  events 

4.  rapidly  changing  technology 

5.  desire  to  identify  business  opportunities 

60.  Why  is  IRM  strategic  planning  important  to  an  organization? 

**  1.  IRM  directly  supports  the  organizational  mission 

2.  All  functions  must  have  a  strategic  plan 

3.  IRM  strategic  plan  is  the  foundation  for  the  organizational  strategic  plan 

4.  IRM  strategic  plan  is  the  primary  input  for  acquisitions  planning 

5.  All  of  the  above 
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